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NASA STI Program. ..in Profile 


Since its founding, the national Aeronautics and Space Administration (NASA) has been 
dedicated to the advancement of aeronautics and space science. The NASA Scientific and 
Technical Information (STI) program plays a key part in helping NASA maintain this important 
role. 

The NASA STI program operates under the auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and disseminates NASA’s STI. The NASA STI 
program provides access to the NASA Aeronautics and Space Database and its public interface, 
the NASA technical report server, thus providing one of the largest collections of aeronautical 
and space science STI in the world. Results are published in both non-NASA channels and by 
NASA in the NASA STI report series, which include the following types: 

• Technical Publications: Reports of completed research or major significant phase of 
research that present the results of NASA programs and include extensive data or theoretical 
analysis. Includes compilations of significant scientific and technical data and information 
deemed to be of continuing reference value. NASA counterpart or peer-reviewed formal 
professional papers but has less stringent limitations on manuscript length and extent of 
graphic presentations. 

• Technical Memorandum: Scientific and technical findings that are preliminary or of 
specialized interest, e.g., quick release reports, working papers, and bibliographies that 
contain minimal annotation. Does not contain extensive analysis. 

• Contractor Report: Scientific and technical findings by NASA-sponsored contractors and 
grantees. 

• Conference Publication: Collected papers from scientific and technical conferences, 
symposia, seminars, or other meetings sponsored or co-sponsored by NASA. 

• Special Publications: Scientific, technical, or historical information from NASA programs, 
projects, and missions, often concerned with subjects having substantial public interest. 

• Technical Translation: English-language translations of foreign scientific and technical 
material pertinent to NASA’s mission. 

Specialized services also include creating custom thesauri, building customized databases, and 
organizing and publishing research results. 

For more information about the NASA STI program, see the following: 

• Access the NASA STI program home page at www.sti.nasa.gov 

• E-mail your question to help@sti.nasa.gov 

• Fax your question to the NASA STI Information Desk at 443-757-5803 

• Phone the NASA STI Information Desk at 443-757-5802 

• Write to: 

NASA STI Information Desk 
NASA Center for Aerospace Information 
7115 Standard Drive 
Hanover, MD 21076-1320 
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1.0 


INTRODUCTION 


1.1 Background 

An Integrated Baseline Review (IBR) is a review of a supplier’s Performance Measurement 
Baseline (PMB). It is conducted by Program/Project Managers and their technical staffs on 
contracts and in-house work requiring compliance with NASA Earned Value Management 
System (EVMS) policy as defined in program/project policy, NPR 7120.5, or in NASA Federal 
Acquisition Regulations. The IBR Handbook may also be of use to those responsible for 
preparing the Terms of Reference for internal project reviews. While risks may be identified 
and actions tracked as a result of the IBR, it is important to note that an IBR cannot be failed. 

The objective of the IBR is to confirm compliance with the following business rules: 

■ The technical scope of work is fully included and consistent with authorizing 
documents 

■ Key schedule milestones are identified 

■ Supporting schedules reflect a logical flow to accomplish the technical work scope 

■ Resources (budgets, facilities, personnel, skills, etc.) are adequate and available for 
the assigned tasks 

■ Tasks are planned and can be measured objectively, relative to technical progress 

■ Underlying PMB rationales are reasonable 

■ Managers have appropriately implemented required management processes 

An assessment of the management reserve with respect to risk not accounted for in the PMB 
should be made as well. 

An IBR is intended to be an extension of the supplier’s existing management practices. 
Anything that does not support the intent of the IBR should be moved outside the review. Risks 
associated with Technical, Schedule, Cost, Resource, and Management Processes that are 
identified during the IBR should be reviewed; action risks should be incorporated into the 
project’s existing risk management process.” 

1.2 Purpose 

This document is intended to be a how-to guide to prepare for, conduct, and close-out an IBR. 
The document lists the steps that should be considered for each IBR, and offers tips for tailoring 
the IBR based on risk, cost, and need for management insight. Appendices to this handbook 
contain samples of documentation typically used in connection with the IBR. Please note that 
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these appendices are samples only, and should be tailored to meet the needs of individual 
projects and contracts. These attachments are available in native format on the NASA EVMS 
Web Site at http://evm.nasa.gov/handbooks.html. . 

While an IBR has traditionally been conducted on contracts, it can be effective when conducted 
on in-house work as well. The same principles, objectives, and processes apply for in-house and 
contract IBR’s; however minor changes may be necessary to the steps in conducting an IBR. 

Appendix A contains an IBR Checklist that can be used as a guide in conducting the IBR and 
ensuring that all steps are considered. 

1.3 Authority 

It is not a requirement to use this document, nor does this document intend to create or levy new 
requirements on a NASA project or contract. It is simply intended to aid customers and 
suppliers preparing for an IBR to understand expectations of the IBR. Following this guide will 
help to ensure that the IBR is successful. Using this document will also help to meet NPR 
7120.5 IBR requirements for in-house programs and projects, as well as the NASA Federal 
Acquisition Regulation Supplement (NFS) 1852.234-2 IBR requirements on NASA contracts. 

1.4 Applicability 

The objectives of an IBR are applicable and beneficial to projects and contracts of all sizes and 
types; however the level of detail and formality of the review vary based on dollar value, risk, 
and need for management insight. The IBR Team Leader should determine if the Decision 
Authority or Technical Authority have prepared a Terms of Reference for the project. The IBR 
should be consistent with the Terms of Reference and assure an adequate level of detailed 
information and analysis is provided to the Decision Authority and Technical Authority. The 
Mission Directorate may impose unique guidelines for the IBR. 

NASA has many reviews during the program and project lifecycles, and some of these reviews 
share common goals and objectives with the IBR. Therefore, when possible, the IBR can be 
combined with these other reviews. It is important, however, to ensure that the intent of the IBR 
still be met and supported by key personnel when reviews are consolidated. 

Appendix B contains a sample format for an IBR for a very small, low-risk project. For a project 
of this size, it is not unusual to have one person act as the Control Account Manager (CAM) for 
all control accounts. In this case, the IBR usually occurs in one day and consists of one team 
conducting all CAM discussions. Often, a presentation format is used to present relevant data to 
the IBR team, and this information should be furnished to the IBR team at least one week before 
the review. 

1.5 Document Structure 

This guide is divided into three major sections: IBR Preparation, On-site IBR, and IBR 
Closeout. 
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2.0 IBR PREPARATION 


2.1 General 

Preparation is the foundation for a successful IBR. The preparation portion of the IBR is the 
most important part of the entire process. Preparation includes the planning that identifies key 
responsibilities, required technical expertise, IBR and/or EVM training, review dates, review 
scope, risk evaluation criteria, documentation needs, disposition of findings, and procedures for 
risk identification, documentation, and incorporation into the project’s risk management 
planning. 


2.2 Determination of Need for an IBR 

The need for an IBR must be determined early on - prior to contract award for contracts and in 
the program and project plans of in-house work. The requirement for contract IBR’s is contained 
in NASA Federal Acquisition Regulation Part 1852. The requirement for in-house IBR’s is 
contained in NPR 7120.5. In addition, the Mission Directorate may establish unique IBR 
requirements and thresholds that must be followed. The Project Plan should identify the 
contracts and in-house work that will require an IBR, including flow-down of IBR requirements 
to major subcontractors. An IBR may also be planned for efforts that do not meet the dollar 
thresholds in 7120.5, but that have significant risk or require more management attention. 
Including the clause at 1852.234-2 in a solicitation will notify potential bidders of NASA’s intent 
to conduct IBR’s. In addition to that clause, or in a case where the clause is not included in the 
RFP or contract, it is a good idea to provide suppliers with more details about the IBR. This will 
ensure that clear expectations are established and provide NASA the ability to tailor IBR 
requirements. Appendix C contains sample IBR language that can be used in solicitations, 
contracts, or even in-house agreements. 


2.3 IBR Planning 

The customer should begin preparation for the IBR should begin as soon as practical after 
determining the need for an IBR. The program or project manager should appoint an IBR 
Coordinator and a Review Facilitator early-on to help facilitate the IBR. The IBR Coordinator 
helps to coordinate various activities to ensure a successful IBR. The Review Facilitator 
provides the team members with earned value management expertise. The roles of the IBR 
Coordinator and Review Facilitator are further described below. 

The first step in planning the IBR is to determine which control accounts will be reviewed at the 
IBR. A Responsibility Assignment Matrix (RAM) that shows the budget separately broken out 
for each control account will help in this assessment. See Appendix D for a sample RAM and 
Section 2.4 for a more detailed description of the RAM. A RAM or equivalent document must 
be requested from the supplier or project. Typically 85% of the dollar value and all of the high 
risk areas should be examined. Usually, at least one Level-Of Effort (LOE) and one material 
account will be included to provide insight into various aspects of the project. The Program or 
Project Manager, with advice from the Review Facilitator, should choose the control accounts 
for the IBR and inform the supplier, usually informally, of the ones selected. 
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Team composition and assignments are based on which control accounts are selected for review. 
Participants should be identified based on their expertise as required for the review. These 
disciplines include program and project management, business management, and technical 
management (e.g., system engineering, software engineering, manufacturing, integration and test 
engineering, and integrated logistics support). When appropriate, the team should include 
subcontractor, Defense Contract Management Agency (DCMA), and Defense Contract Audit 
Agency (DCAA) personnel. There may be several sub-teams with discussions scheduled 
concurrently, or one team can interview every CAM. Each of these teams should be assigned a 
sub-team leader to lead the discussion. To be effective, the discussion group should remain 
small and focused, usually 3-4 people per team. Typically, the technical experts for NASA will 
lead the discussion. A sample team may consist of an EVMS analyst, a Project Manager, and 
one or two technical experts in the area of the Control Account (CA) to be discussed. A good 
rule of thumb is to allow at least two hours to conduct the CAM discussions and one hour for 
completing documentation. 


2.4 Customer Team Roles and Responsibilities 

Program or Project Manager (PM): The PM either acts as or assigns the Team Leader for the 
IBR. The overall responsibility of conducting the IBR lies with the Team Leader. The Team 
Leader is responsible for coordinating the activities of all individuals assigned to perform the 
review. Some individual tasks to be performed by the Team Leader are described below. 

a. Coordinate with the supplier to discuss the IBR strategy, areas to be reviewed, and 
tentative dates for the IBR. 

b. Choose appropriate team members and assign responsibilities. 

c. Ensure team members are adequately trained and prepared. Provide IBR training and 
workshops as necessary to prepare the team before the review begins. 

d. Provide technical direction and leadership emphasizing the importance of thorough 
cost, schedule and technical integration of work. 

e. Provide an in-brief and out-brief at the IBR. 

f. Provide an overall evaluation of the IBR and assess whether or not a follow-on 
review will be required. 

g. Document review findings. The format of this documentation is at the discretion of 
the Team Leader. Sample formats are included as part of the IBR Guidance Package 
in Appendix F. 

h. Ensure that all issues identified during the review are resolved in an appropriate and 
timely manner. Assign action items and completion dates to appropriate government 
and supplier personnel. Schedule action item review as necessary. 

i. Define the risk evaluation criteria to generally categorize the risk that may be 
identified at an IBR: cost, schedule, technical, resource, and management processes. 
Appendix F, the IBR Guidance Package, contains sample criteria for each type of 
risk. 
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IBR Coordinator: The Team Leader may appoint an individual as IBR Coordinator whose 
function entails coordination of various activities to ensure a successful Integrated Baseline 
Review. This includes but is not limited to coordinating review dates, obtaining information 
necessary for the review, organizing this material, and coordinating the various aspects among 
the government and supplier review participants. The following is a recommended guide to 
assist the coordinator in performing this role. 

a. Coordinate the review dates that are amenable to all parties. Consideration should be 
given to a time period that is the least disruptive to ongoing activities. Normally 
scheduling the review in the early part of the month after the close of the accounting 
period and statusing activities is the most appropriate time. 

b. Provide written notification of the IBR to the supplier. Send the letter to all of the 
team members for informational purposes. 

c. Obtain a list of the IBR Team members from the Team Leader. 

d. Coordinate with individual team members to ensure travel and security arrangements 
have been accomplished. 

e. Prepare the IBR Guidance Package. 

f. Collect information and data requested in the IBR notification letter; compile and sort 
the information by sub-team responsibilities; and make this information, along with 
the IBR Guidance Package, available to the respective sub-team leads. 

g. Schedule an IBR Kick-Off meeting to allow team members to familiarize themselves 
with the documentation and develop questions to be addressed during the review. 

h. Collect the IBR Concern Area Report Forms or questions that have been generated 
during the pre-IBR meeting for the Team Leader’s review. Forward those deemed 
appropriate by the Team Leader to the supplier for resolution. The IBR Coordinator 
will perform this function during the on-site review also. 

i. Coordinate with the Team Leader to ensure all team members receive necessary IBR 
training prior to the review. 

j. Distribute IBR forms, either hard-copy or electronic format, to team members prior to 
CAM discussions. 

k. Collect documentation at the IBR. 

l. Establish and maintain an IBR log. 

Review Facilitator: Duties and responsibilities of the IBR Review Facilitator include 
providing the team members with earned value management expertise. This is accomplished 
through the review of documentation, assessing earned value status, and interpreting issues 
that relate to overall earned value management. The Review Facilitator should be an EVM 
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Working Group member from the Center if possible. The items listed below are provided as a 
recommended guide to assist the facilitator in the accomplishment of this role. 

a. Determine status of the EYM system. If applicable, request documentation showing 
that the EVM system has been previously accepted by the government. When 
applicable, contact the local DCMA representative to determine if there are any 
outstanding earned value management system problems that would affect the quality 
of the performance measurement data. 

b. If the EVM system requires previous government acceptance and has not met that 
requirement, coordinate with appropriate personnel the means by which this 
acceptance can be accomplished. 

c. Review management processes that will be used in the management of the work. 

d. Provide earned value management expertise to the IBR Team by assisting team 
members in the understanding of any EVMS data obtained prior to the review. 

e. Coordinate with the Team Leader and IBR Coordinator in determining the interview 
schedule. 

f. Review the experience and background of the team participants and provide a 
recommendation to the team leader of the type and amount of EV training deemed 
necessary. 

g. Provide assistance to IBR Team members in determining whether individual tasks 
have been assigned appropriate earned value techniques used for measuring progress. 

h. Assist the team leader in the development of the risk evaluation criteria. 

i. The IBR Facilitator reviews all action items generated to ensure that earned value 
issues requiring follow-up are appropriate and necessary. 

j. Provide assistance to the Team Leader with the preparation of both the in-briefing 
and out-briefing at the IBR. 


Sub-team Technical Lead: The Sub-team Technical Lead is normally a technical expert who 
specializes in the area that is being addressed during the CAM discussion. The responsibilities 
of the Sub-team Technical Lead include: 

a. Attend the pre-IBR meeting, including training and documentation review prior to the 
start of the IBR. 

b. Review documentation prior to baseline discussions with the CAM. 

c. Lead CAM and senior manager discussions. 

d. Establish the strategy for conducting the CAM discussions, e.g., designates a single 
person to lead the discussion, someone to take notes, etc. 
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e. Ensure all applicable documentation is properly completed (i.e. Discussion 
Assessment Form, Concern Area Report, Documentation Request Form, and Risk 
Assessment Form). 

f. Provide daily out-briefs to keep the IBR Team informed. 

g. Provide an assessment of risk based on the prescribed risk evaluation criteria. 

h. Assist in the preparation of the IBR out-brief. 

Technical Subject Matter Expert (SME) Team Members: Team members may include 
technical experts, EVM analysts, procurement representatives, DCMA, as well as other 
personnel who may be of benefit during the review. Duties of team members include: 

a. Attend the pre-IBR meeting, including training and documentation review, prior to 
the start of the IBR. 

b. Review documentation prior to baseline discussions with the CAM. 

c. Conduct CAM and senior manager discussions. 

d. Assist in completing all applicable documentation. 

e. Provide an assessment of risk based on the prescribed risk evaluation criteria. 

f. Assist in completing IBR documentation. 

g. Assist in the preparation of the IBR out-brief. 

Schedule Analyst: The duties of the Schedule Analyst include: 

a. Attend the pre-IBR meeting, including training and documentation review prior to the 
start of the IBR. 

b. Review documentation prior to baseline discussions with the CAM. 

c. Conduct CAM and senior manager discussions. 

d. Review the EVM Systems Description or management processes to understand the 
processes and procedures by which the Integrated Master Schedule is created and 
maintained. 

e. Provide an assessment of all project risk based on the defined risk evaluation criteria. 

f. Assist in completing all IBR documentation. 

g. Assist in the preparation of the IBR out-brief. 
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Cost Analyst: The responsibilities of the Cost Analyst include: 

a. Attend the pre-IBR meeting, including training and documentation review prior to the 
start of the IBR. 

b. Review documentation prior to baseline discussions with the CAM. 

c. Provide insight into the reliability of the Basis Of Estimate. 

d. Inform the team of any areas where the independent or non-advocate estimates differ 
from the advocate estimate. 

e. Conduct CAM and senior manager discussions. 

f. Provide an assessment of all project risk based on the defined risk evaluation criteria. 

g. Assist in completing all IBR documentation. 

h. Assist in the preparation of the IBR out-brief. 


Having a representative from the Independent Program Assessment Office (IPAO) and/or 
Systems Management Office (SMO)on the IBR team is recommended to ensure an independent 
member of the review panel. The team member will provide a common thread for all reviews at 
a particular Center to ensure consistency across all the reviews and provide an avenue for lessons 
learned to be communicated across the Center. This will also familiarize the SMO or IPAO 
representative with the project to make independent evaluations that occur later in the project life 
cycle more thoroughly and efficiently conducted and completed. 

It is also important to include trainees and observers as part of the IBR process. This is a good 
way to provide training to people who may need to participate in a future IBR. 


2.5 Determine Supplier Readiness for the IBR 

Prior to the IBR on-site review, the IBR Team must assess the readiness of the supplier to 
conduct an IBR. Below are some rules of thumb to help determine IBR readiness. 

■ The PMB should reflect the entire scope of work. 

■ The schedules reflect the entire scope of work. 

■ It is good to have at least two reporting cycles worth of data before the IBR. This 
will provide the IBR Team with some performance measurement data as well as a 
feel for the accuracy of the data. 

■ Look for detail planning to the work package level for six months to one year to 
ensure an adequate assessment of the baseline. 
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The IBR Team must measure the ability of the supplier to meet the intended objectives prior to 
conducting the IBR on-site review. The IBR is not just a check in the box! If after the IBR, the 
Program or Project Manager does not feel that the IBR met the objectives, a second IBR must be 
performed. 


2.6 IBR Notification Letter/Documentation 

An IBR notification letter should be prepared and sent to the supplier within 3 months of the 
scheduled on-site IBR. This notification letter can take any form but should include the 
following information: the purpose of the review, the time frame of the review, the agenda, 
documentation requested, team requirements (ex: fax and phone access, meeting rooms, etc), 
IBR Team members and roles, and points of contact for follow-up. See Appendix E for a sample 
IBR notification letter. For contracts and subcontracts, the IBR Notification Letter should be 
sent from the government Contract Technical Representative to the contractor organization. The 
local DCMA and DCAA organization, if applicable, should be informed of the review. 

One of the main functions of the notification letter is to request documentation for use on-site 
and prior to the review so that the team can prepare. This documentation will be used by the 
team to identify risk areas and to develop preliminary questions for the on-site review, and 
should be received 2-4 weeks prior to the IBR. The following is a listing of some of the 
documentation that may be requested. 

a. Organizational Breakdown Structure (OBS) 

b. Control Account Plans (CAP): Basis of Estimates (BOE), Assumptions, and Risk 

c. Work Authorization Documents (all levels) 

d. Performance Measurement Reports (internal cost/schedule reports) 

e. Estimate at Completion (EAC) 

f. Budget Change Requests (BCR): 

g. Control Account (CA)AVork Package (WP) summary containing: 

■ Number of work packages by type of EVM method 

■ Longest CA, shortest CA, mean and median duration, total value of account 

■ Largest CA, smallest CA, mean and median values 

h. Baseline Control Logs (Management Reserve, Undistributed Budget, Contract Budget 
Base) 

i. SOW with Work Breakdown Structure (WBS) Matrix 

j. Schedules (Master, Intermediate, and Detailed) 

k. EYM System Description 

l. Description of Management Process such as baseline maintenance, risk management, and 
other business processes 

m. Financial Management Reports such as 533M/Q or equivalent 

n. All Contract Performance Reports (CPR) submitted to date 

o. Responsibility Assignment Matrix (RAM) 

p. WBS or Contract Work Breakdown Structure (CWBS) and Dictionary 

q. Subcontractor listing and value of subcontracts 

r. All authorized changes 

s. Contractor/Subcontractor Flow-Down Requirements 

t. Risk Register from Risk Management System 
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Only request data that has not already been received. Also, this list should be tailored based on 
information required for preparation and insight. The supplier may have the data described 
above in different formats, and these documents are acceptable and even preferred as 
replacements when the information is sufficient. 

Responsibility Assignment Matrix (RAM): 

The RAM correlates the work required by a WBS or CWBS element to the functional 
organization responsible for accomplishing the assigned tasks. The RAM results from the 
intersection of the WBS or CWBS with the organizational structure. This intersection identifies 
the CA. The RAM includes the functional organization and the individual responsible for the 
CA, known as the CAM as well as the budget for each CA. 

Work Authorization Documents: 

All work, regardless of origin, is described and authorized through the work authorization 
system. Work authorization assures that performing organizations are specifically informed 
regarding their work scope, schedules for performance, and budget for that work. 

Work authorization is a formal process related to various levels of the organization. Agreement 
is reached to each level of authorization so that there is no question as to what is required. The 
CA authorization represents an agreement between the CAM and the authorizing party. The 
documents involved in work authorization, at each level, are maintained in a current status as 
revisions take place. 

Cost/Control Account Plans (CAP): 

A CAP is a plan of all effort to be performed in a cost/control account. The CA is the focus for 
planning, monitoring, and controlling because it represents work within a single WBS or CWBS 
element, and it is the responsibility of a single organizational unit or work team. Virtually all 
aspects of a project come together at the CA level, including budgets, schedules, work 
assignments, cost collection, progress assessment, problem identification, corrective actions, and 
EAC development. 

A Work Package (WP) is a detailed job that is established by the CAM for accomplishing work 
within a CA. A WP can be characterized by the following: 

a. The WP represents units of work at the levels where the work is performed. 

b. A WP is clearly distinct from all other WP’s, and is the responsibility of a single 
organization element. 

c. A WP has scheduled start and completion dates (with interim milestones, if applicable), 
which are representative of physical accomplishment. 

d. A WP has a budget or assigned value expressed in terms of dollars, labor hours, or other 
measurable units. 

e. The duration of a WP is relatively short, unless it is subdivided into discrete value 
milestones that permit objective measurement of work performed. 
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f. The WP schedule is integrated with all other schedules. 

g. The WP has a unique earned value technique for determining the value of completed and 
in-process work. 

If a CA cannot be subdivided into detailed WP’s, far-term effort may be identified in larger 
Planning Packages (PP’s) for baseline control purposes. The budget for a PP is identified 
specifically for the work that is intended, is time-phased to the extent possible, and has controls 
that prevent its use in performance of other work. Eventually, all work in PP’s will be planned to 
the appropriate level of detail in WP’s. 

Integrated Master Schedule (IMS) 

Various schedules are essential to review before IBR discussions begin. 

a. Integrated Master Schedule 

The Integrated Master Schedule is the controlling schedule for the entire contract or in- 
house work activity. It contains required milestones/activities. An integrated schedule 
developed by logically networking all detailed activities. The highest level schedule is 
the Master Schedule supported by Intermediate Level Schedules and by lowest level 
detail schedules. 

b. Intermediate Schedule(s) 

Intermediate Schedules are the integrating schedules for a contract or in-house work 
activity. There may be single or multiple levels of intermediate schedules. They may be 
oriented to either WBS/CWBS elements or to functional organizations. In either case, 
the purpose is to form a bridge between the Master Schedule and CA schedules. The 
term “integrating” is used since the purpose is to coordinate the effort of a number of 
CA’s that are related in one way or another to the scope of the intermediate schedule. 
Intermediate schedules are often the primary tool for identifying key interfaces through 
network logic. 

c. Cost/Control Account (Detail) Schedules 
See Cost/Control Account Plans. 

Electronic versions, in native vile formats, are necessary for providing insight into the various 
schedules. 

Cost Reports: 

As part of the documentation review process, it is important to review the supplier’s latest cost 
report deliverables, i.e. 533M or 533Q, etc. 


17 



WBS/CWBS and Dictionary: 


The Work Breakdown Structure or Contract WBS is the basis for further extension to lower 
levels that represent how the supplier plans to accomplish the entire work scope and is consistent 
with internal organizations and processes. This extended CWBS or WBS serves as the 
framework for reporting of cost and schedule status. The WBS or CWBS Dictionary is used to 
describe the effort and tasks associated with corresponding elements. 

Once the documentation is received the next step is for the IBR Team to evaluate this 
documentation and determine risk areas. This evaluation of documentation, as well as other 
activities, can be conducted at a pre-IBR meeting. Questions may be held for discussion at the 
IBR or can be provided to the supplier prior to the IBR, allowing time to respond prior to the 
IBR team arriving at the facility. The questions can be informal or documented in a Concern 
Area Report (CAR). See Appendix F for a sample CAR. If the documentation review uncovers 
new risk areas, those control accounts should be added to the review schedule. 


2.7 IBR Guidance Package 

An IBR Guidance Package should be prepared that serves as a guide for the IBR Team. The IBR 
Guidance Package will assist the team members in conducting the review. The package provides 
the contract or in-house work background, sample discussion questions, documentation 
guidelines, sample documentation, risk evaluation criteria, and a glossary of EVMS terms. The 
review agenda and team assignments should also be included in the package. See Appendix F 
for a sample IBR Guidance Package. 


2.8 Training 

Training should be conducted at a Pre-IBR meeting. The three components of the training are 
basic EYM training, IBR training, and an overview by the supplier. The supplier can be a great 
help during the Pre-IBR meeting, by explaining documentation and answering any preliminary 
team questions. The supplier personnel can also give the team an understanding of management 
processes, such as baseline maintenance, risk management, and other business processes 
including EYM, that will be used to manage the project. Note that while EVMS processes are 
discussed at an IBR, the IBR is not an EVMS Validation Review. The team will also review the 
IBR Guidance Package during the training, and learn the use of the various forms and 
assessments used to document interview findings. 

Training may be obtained from various sources and should be tailored based upon the needs and 
experience of the individual members of the review team. Training sources may be found by 
contacting the local NASA Training Coordinator or the NASA EVM Working Group. Names of 
members and contact information are listed on the NASA EVM Website. 

If at all possible, the training and documentation review should occur prior to the on-site IBR. 
This will help to ensure a smoother review; provide the team an opportunity to review the data 
for completeness; and ensure readiness of the supplier. If the team is not able to have ample 
training and preparation time prior to the on-site review, sufficient time needs to be allocated to 
accomplish these efforts. Conducting the training and documentation review on-site 
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immediately followed by the CAM discussions may also be done. If this is done, make sure to 
allow adequate time for the team members to complete a review of the documentation prior to 
the start of the IBR discussions. See Appendix G for sample IBR and EVM Training. 


2. 9 Travel Arrangements/IBR Logistics 

Team members need to be informed as early as possible of the travel plans, of the review 
schedule and agenda, and of their assignments. Additionally, team members should know the 
name and number of a POC for the IBR, and security arrangements need to be made for each 
team member visiting the facility (i.e. Visit Requests need to be sent to the Security Office.) IBR 
Team members should also be provided area maps, directions to facilities, directions to visitor 
check-in, and specific directions to the building and room where the initial meeting or in-briefing 
will be held. 
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3.0 ON-SITE IBR 


3.1 General 

The preparation and planning done up to this point pay off at the on-site IBR. The primary 
purpose is for the Program and Project Managers to gain a mutual understanding of the risks 
inherent in the PMB and management processes. Anything that does not support this purpose 
should be discussed outside of the IBR. Any issues that arise during the IBR should be recorded, 
but resolved after the review. 

The intent of this section is to provide sample tools and documentation for conducting the on-site 
IBR. While each IBR may be tailored, these tools will provide a starting point for conducting 
the review. 


3.2 On-site Discussions 

Once the IBR Team is on-site at the supplier’s facility, several different activities must be 
considered. This section describes of each of these activities and their role in making the IBR 
successful. See Appendix H for a review in-brief template. 


3.2.1 In-Briefs 

3.2.1. 1 Joint IBR Team and Supplier 

In some cases the IBR team may want to provide an in-brief to the supplier at the start of the 
IBR. The main theme of this discussion is to re-emphasize that the purpose of the IBR is to 
evaluate the adequacy of the baseline and identify concerns, not to try to solve problems. 


3.2.1.2 IBR Team-Only 

The team may also want to conduct an IBR Team in-brief before the CAM discussions begin. 
This meeting is the final opportunity to focus the IBR Team on the objectives. 


3.2.2 CAM Discussions 

The CAM discussions are the key events of the IBR. These discussions focus on key risk areas 
and management processes. During this period the team members should cover the key aspects 
of the planning of the work scope. Overall success of the IBR results, in part, through productive 
CAM discussions. Following the techniques and formats below will help to ensure that adequate 
information is obtained in a timely manner to accomplish the objectives of the IBR. 


3.2.2.1 CAM Discussion Guidelines 

Below is a list of suggested techniques for all team members to consider before and during the 
CAM discussions. 
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a. Prior to the discussions, the IBR Team members should be familiar with areas 
previously identified for discussion. Review the documentation thoroughly. 

b. Prior to the discussion, conduct a basic data and documentation trace to become 
comfortable with the data and how it flows through the system. Appendix I contains 
a documentation and data trace format that may be useful in conducting the trace. 

c. Introduce yourself and identify the organization you represent. You may also wish to 
indicate your team affiliation in the review. 

d. Be well prepared and maintain a tempo that keeps the discussion moving along 
toward satisfying your objective. Be friendly, but avoid long conversations 
extraneous to the discussion. 

e. Take notes. The discussion leader (normally the Sub-team Technical Lead) or an 
accompanying team member should take notes during the discussion. 

f. Request copies of documents only if necessary to accomplish the objective of the 
discussion. If documentation is not readily available, complete a Documentation 
Request Form and submit it to the IBR Coordinator. See Appendix F for a sample 
Documentation Request Form. 

g. Watch the time. Discussions are normally scheduled for two hours in length. Should 
additional time be required to complete the discussion, coordinate with the Team 
Leader 

h. If disagreements arise which cannot be resolved, the team member should write a 
description of the disagreement in a Concern Area Report and submit it to the Team 
Leader for disposition. The Team Leader will handle any continuing discussion. 

The supplier must ensure that each CAM is well prepared (documentation available, 
understands documentation content, can support answers, etc.). 

3.2.2.2 Discussion Topics 

Below are some examples of information that should be obtained from the CAM discussions. 

This list of questions is not all-inclusive and should be tailored for each project. Additional 

sample questions can be found in the IBR Guidance Package in Appendix F. 


a. What is the Manager’s scope of work? Does the Manager know what scope of work 
is his/her responsibility? 

The Manager should be able to show the team a Statement of Work (SOW) 
paragraph, a WBS, or CWBS narrative, and a Work Authorization Document that 
correlates to the task. 

b. How did the Manager plan the scope of work into control accounts? 
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The SOW defines the effort. The WBS or CWBS provides specifics, such as work 
definition. The work authorization and change documentation should show 
information such as the dollars/hours, period of performance, and description of the 
scope of work and any changes. 

c. How did the Manager ensure that all elements of the scope of work were planned? 

The Manager should be able to show the team the work packages breaking down the 
scope of work, and the budget associated with each work package. The sum of the 
work packages and planning packages should be the control account budget. Each 
control account should contain a specific scope of work. 

d. How did the Manager obtain the resources for assigned work? 

This information should be obtained through the Work Authorization Document. The 
work authorization should provide specific resources used to manage the scope of 
work. 

e. How did the Manager come to an agreement of the scope, hours, dollars and 
schedule? 

f. Ask the Manager to show the team the schedule milestones used in the planning of 
the control accounts. The Manager should discuss: 

■ Schedule impacts 

■ Schedule impacts from other organizations 

■ Resource levels which support schedule milestones 

■ Schedule constraints 

■ Relationships to other tasks 

■ Logical relationships of work package to milestones 

■ Level of Effort (LOE) tasks that support the schedule 

■ The Manager should be able to show the team the activity supported on the 
appropriate schedule 

■ Critical path items for which the CAM is responsible 

h. Ask the Manager how the work was time-phased to achieve the schedule. 

All work should be logically planned in accordance with the scope of work and the 
schedule. 

i. Ask the Manager to identify any known risks that may affect the plan. 

j. Are the Manager's resources adequately planned and correctly time phased to meet 
the plan? Progress of each plan should relate to earned value. 
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k. Ask the Manager why he/she chose a particular type of earned value method. 

Does the earned value method used make sense for the type of work being 
performed? Can you measure performance? Is the Manager using milestones to 
measure accomplishments? Does the Manager use the LOE method; and is the LOE 
in support of measurable scope? Is LOE the appropriate earned value method for the 
task? 

l. How does the Manager manage his/her plan? 

Does the manager use monthly reports, latest schedules, technical assessment of 
impacts, daily meetings, and continuous assessment of the plan? 

m. How is the baseline maintained? 

How does the CAM ensure changes are incorporated into the baseline? What are the 
controls (retroactive, freeze period, etc.) to ensure the integrity of the baseline? 

n. What is the risk management process? How is risk incorporated into the EAC? 

o. What is the process for reviewing and updating EAC’s? 

p. How does the CAM update the schedule? How often are schedules updated? 


3.2.2.3 IBR Documentation and Forms 

After each CAM discussion, each Sub-team should sit down and review what was discussed. 
This is an opportune time to collect documentation to better prepare for the out-brief 
presentation. Each Sub-team Leader is responsible for documenting the team’s assessments and 
findings. The Team Leader uses the documentation to support overall team assessments and 
required corrective actions. 

The following forms have been developed to facilitate review documentation. Sample IBR 
forms are provided in the IBR Guidance Package in Appendix F. Note that these are just sample 
formats and program/project specific formats can be substituted. 

Discussion Assessment Form: 

The Sub-team should complete one of these forms after each discussion. The Sub-team 
Technical Leader is responsible for reviewing this form and submitting it to the Team 
Leader and the IBR Coordinator. Keep in mind that the Discussion Assessment Form 
can be tailored based on risk. 

Concern Area Report (CAR): 

A CAR should be completed for each concern noted. Generally, concerns noted are 
items that require follow-up action. Information on this form should be clear and as 
specific as possible because it will be provided to the supplier to obtain a response and 
resolution. Spell out abbreviations. These forms should be submitted immediately to the 
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Team Leader, and as soon as practical to the supplier. Responses should be provided to 
the IBR Team as soon as possible. Again, the Sub-team Leader is responsible for 
reviewing this form, and submitting it to the Team Leader and the IBR Coordinator. All 
forms will be compiled and logged. 

Documentation Request Form: 

During CAM discussions, the team may find that additional documentation is required to 
gain a better understanding of the issue in question. Use this form to obtain required 
documentation. Submit the completed form to the Team Leader and the IBR 
Coordinator. All forms will be tracked in order to reduce redundancies and ensure 
receipt of all the requested material. 

Risk Assessment Form: 

A risk assessment form should be completed after each discussion. Each risk and 
opportunity identified during the CAM Discussion is assigned a probability of 
occurrence. A potential cost and schedule impact for each risk and opportunity should be 
estimated, and a determination made as to whether this risk has been accounted for in the 
baseline. Each risk is classified as cost, schedule, technical, resource, or management 
process and assigned a rating based on risk evaluation criteria established by the NASA 
Program or Project Manager prior to the IBR. The ultimate goal of the Risk Assessment 
area of the IBR is an updated Estimate At Complete (EAC) or Life Cycle Cost estimate 
(LCC) which incorporates quantified risks. The Sub-team Leader is responsible for 
reviewing this form and submitting it to the Team Leader and IBR Coordinator. Risks 
identified at the IBR should be incorporated into the project’s existing risk management 
plan. 

General Assessment Form: 

The form is used to document observations for those things that are “not CAR worthy”, 
but might help EYM efficiencies (such as automatic links between IMS and EVM 
engine), as well as potential problem areas in the future based on previous IBR lessons 
learned. It is also used to document “best practices” for those things that improve the 
EVM process. 


All forms should be compiled and logged in an IBR Log. The IBR log should consist of both 
potential and actual issues revealed prior to and during the review, and should contain the 
appropriate actions required along with anticipated completion dates. This IBR Log will support 
the Letter of Findings during the IBR Close-Out Phase discussed in Section 4. See Appendix J 
for sample IBR Logs. 


3.2.3 Out-briefs 

3.2.3. 1 Daily Team Out-briefs 

At the end of each day, the NASA IBR Team should conduct out-briefs where each sub-team 
reports findings from the CAM discussions. This will ensure that the entire team is aware of 
issues that have been raised by other teams, allowing further investigation during other CAM 
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discussions. A supplier representative can be a part of this meeting, helping to clarify any issues 
and resolve issues in a timely manner. 

3.2.3.2 Final IBR Out-brief 

An IBR team meeting should be conducted prior to the final out-brief where the IBR Team can 
discuss the brief and concerns. This will ensure that the IBR Team members have a clear 
understanding of any concerns and action items. The agenda for the IBR must ensure that ample 
time is made available to prepare the out brief presentation. 

The Team Leader should provide an out-brief to the supplier at the end of the on-site IBR. The 
out-brief should be tailored for each contract or in-house work activity; however several areas 
should be included. First, the brief should include assessments of each area’s PMB discussed 
(pros and cons) as well as all Concerns and Action Items. Next, a summary of the overall risk 
should be included as part of the out-brief as well. Finally, the Team Leader should provide an 
overall evaluation of the IBR and assess whether or not a follow-on review will be required. 

This decision is based on the Team Leader’s assessment of whether the objectives of the IBR 
have been achieved. 

A sample IBR Out-Brief can be found in Appendix K. 

The Team Leader may also want to have the Sub-team Technical Leaders brief their area(s) of 
expertise. In addition, the Review Facilitator should note to the team any concerns and issues 
related to the management processes. 
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4.0 IBR CLOSEOUT 


4.1 General 

Once the IBR out-brief is completed, the closeout portion of the IBR process entails the review 
of all corrective actions identified during the IBR and the establishment of a plan to formally 
close out the IBR. The Program and Project Managers should agree on the closure plan of action 
and identify the individual(s) responsible for all identified risks. This phase also includes 
capturing lessons learned from the IBR process. 


4.2 Letter of Findings/IBR Report 

The results of the IBR should be reported to the supplier, and an IBR Report or a Letter of 
Findings can be used to effectively communicate these results. The IBR is an event required by 
the contract, therefore a letter to the contractor issued by the Contract Technical Representative 
(COTR) is customarily sent. The Review Facilitator should work with the Team Leader to 
generate the letter to the supplier. A letter will help to prevent the supplier from claiming that 
resolving actions resulting from the IBR are out of scope. 

The purpose of the letter or report is to summarize all of the concerns that were found during the 
IBR discussions and to request a corrective action plan from the supplier. The corrective action 
plan should identify proposed corrective/preventative actions, responsibility assignments, and 
projected completion dates. Issues that could impact the performance measurement data should 
also be identified and a copy provided to the appropriate EVM Working Group member. 
Appendix L contains a template for a Letter of Findings and Appendix M contains an IBR 
Report format. The IBR Concern Area reports or a Concern Area summary spreadsheet would 
typically be acceptable attachments to the letter. Also, if an IBR Report is prepared, it should be 
included as part of the letter to the supplier. 

As an alternative, the customer and the supplier could agree on a corrective action plan 
immediately after the IBR out-brief to ensure more timely resolution of issues. This will make 
certain that the concerns are understood and that the IBR team agrees with the corresponding 
corrective action plan. This can speed resolution of the Concern Area Reports and closeout of 
the IBR. 


4.3 Tracking Concerns 

Tracking the progress in resolving each concern rests with the Team Leader, including the 
Defense Contract Management Agency (DCMA) team representatives located at the contractor’s 
facility when applicable. The IBR Coordinator should work closely with the IBR Team to 
ensure that all actions are closed. Regular project reviews are also another good place to have 
the follow-up on any actions that resulted from the review. Approval of corrective actions rests 
with the Team Leader, and most likely would be an iterative process. 
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4.4 IBR Closeout Letter 

When all Concern Area Reports have been closed, the Team Leader will approve close out of the 
IBR. An IBR closeout letter is sent to the supplier indicating that all CAR’s are either closed or 
being tracked and that the IBR is complete. Reporting the results of the IBR may take the form 
of an informal letter or memo for record, depending on the size and formality of the IBR. See 
Appendix N for a sample IBR Closure Letter. 


4.5 Lessons Learned 

After the IBR on-site review and actions have been agreed upon, it is important to poll the IBR 
participants for lessons learned. This includes the areas that worked well and those where 
improvements could be made. Lessons learned can come from both the IBR team and the 
supplier team. Lessons learned should be forwarded to the Agency’s Program Executive for 
EVM as the Agency EVM Working Group is responsible for the Agency IBR Toolkit. This 
document was developed using best practices from previous IBR’s, and therefore these are 
included within the document. In addition, below is a listing of some additional lessons learned 
from both in-house and contractor IBR’s. This list is updated as IBR’s occur. 

a. If a form includes an assessment of CAM discussions, ensure that criteria for those 
ratings is established, clear, and objective. If an unfavorable rating is assessed, include 
rationale supporting the rating. 

b. The team leader should discuss interview strategy with the team members prior to the 
interview to describe conduct of the review, i.e. who will ask questions, who will take 
notes, etc. This will ensure that team members understand roles and responsibilities 
during the discussions. 

c. Supplier participation in the daily out-briefs can help to ensure that concerns are clearly 
communicated and understood. However, it is important to stress the purpose of the 
meeting is for team discussion of issues and not for resolution of those issues. 

d. CAM discussions should not be scheduled for less than two hours to ensure that teams 
have enough information to answer questions on discussion and concern area forms. 
Ample time must also be allowed to poll all team members. 

e. If the time allotted for CAM discussions is insufficient, work with the CAM to schedule 
additional time. This will eliminate the need for side-bar discussions, ensuring that the 
whole team is aware of the information. 

f. Dedicate a room for the IBR team for the duration of the IBR. 

g. Provide hard copies of logistics information (i.e. schedule, maps, etc.) to IBR team 
members as part of the in-brief on the first day of the on-site IBR. 
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APPENDIX A IBR CHECKLIST 


IBR Event: 
IBR Date: 


Phase 1 - Organization & Planning 


TARGET DATE 

DONE 

RESPONSIBILITY 

REMARKS 

Determine Need for IBR 



Project Manager 


Appoint IBR Coordinator and Review Facilitator 



Project Manager 


IBR Overview training for PM & leadership 
team 



Facilitator 


Define CWBS/CA scope for IBR 



Team Lead 


Request Responsibility Assignment Matrix 
(RAM) from Supplier 



Team Lead 


Determine Supplier IBR Readiness 



Team Lead 


Determine areas to be reviewed at the IBR 



Team Lead 


Identify & Assign Review Team members 



Team Lead 


Coordinate IBR dates and review agenda with 
the supplier and Program/Project Mgmt 



Coordinator 


Formally notify supplier of Review & request 
project data & documentation 



Coordinator 


Create a Common Data Repository/Archive site 
for IBR data drop 



Coordinator 


Collect requested project documentation and 
distribute to IBR Team 



Coordinator 


Define Risk Evaluation Criteria 



Team Lead 


Prepare IBR Guidance Package 



Coordinator 


Phase II - Prepare & Train for the IBR 


TARGET DATE 

DONE 

RESPONSIBILITY 

REMARKS 

Conduct IBR Kick-off Meeting 



Team Lead 


Provide training for Team Members 



Team Lead 


Plan for Team travel and logistics for the On- 
site Review 



Coordinator 


Coordinate with contractor/project team for 
On-site support required 



Coordinator 


Collect concerns from the IBR Kick-Off Meeting 
and forward to supplier 



Coordinator 


Determine Discussion Questions 



Sub-team Lead 


Phase III - On-Site Review Activities 


TARGET DATE 

DONE 

RESPONSIBILITY 

REMARKS 

Assemble Team on-site to review IBR plan and 
schedule 



Team Lead 


Deliver IBR In-Brief at Kick-Off Meeting 



Team Lead 


Conduct summary-level planning reviews and 
discussions/interviews 



Sub-team Lead 

Per interview or per day activities. 
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Conduct detailed (CA level) 
discussions/interviews 



Sub-team Lead 


Hold daily end-of-day Team meetings to review 
issues/findings 



Team Lead 


Prepare assessment Reports for each 
discussion/interview 



Sub-team Lead 


Identify and request any additional data or 
documentation required 



Coordinator 


Document concerns using the Concern Area 
Report (CAR) template 



Sub-team Lead 


Document action items using the IBR Action 
Items template 



Sub-team Lead 


Assign action items and completion dates to 
appropriate people 



Team Lead 


Prepare final IBR Out-Brief and review with 
senior management 



Team Lead 


Deliver final Out-Brief 



Team Lead 


Phase IV - Follow-up Activities 


TARGET DATE 

DONE 

RESPONSIBILITY 

REMARKS 

Formally notify Supplier of IBR Findings 



Coordinator 


Monitor action item progress 



Team Lead 


Notify Supplier of IBR Closure 



Coordinator 


Schedule further IBRs as appropriate over the 
life of the contract/project 



Team Lead 
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APPENDIX B 


SMALL PROJECT IBR 



Integrated Baseline Review (IBR) 
Project Name 

Project Manager’s Name 
Date 



WBS/Work Description/Reauirements 




RAC ($K) 

Work 

D e s Clip! ion 

Requirement 

Paragraph 

1 

Space System 

3185 

3.0 


1.1 


Launch "Vehicle 

510 

3.1.1 

1 

1.1.1 



Propulsion (Single Stage Only) 

150 

3.1. 1.1 

2 

1.1.2 



Stage I 

120 

3.1. 1.2 

3 

1.1.3 



Stage II. ..n (As Required) 

150 

3.1. 1.2 

3 

1.1.4 



Guidance and Control 

60 

3.1. 1.3 

4 

1.1.5 



Integration. Assembly, Test and 
Checkout 

30 

3. 1.1.4 

5 

1.2 


Orbital Transfer Vehicle 

590 

3.2.2 

6 

1.2.1 



Propulsion (Single Stage Only) 

220 

3.2.2. 1 

7 

1.2.2 



Stage I 

75 

3.2.2. 1 

8 

1.2.3 



Stage II... n (As Required) 

lOO 

3. 2. 2. 2 

9 

1.2.4 



Guidance and Control 

150 

3.2.2.3 

9 

1.2.5 



Integration, Assembly, Test and 
Checkout 

45 

3. 2. 2. 4 

IO 

1.3 


Space Vehicle 


1650 

3.2.3 

11 

1.3.1 



Spacecraft 

600 

3.2.3. 1 

12 

1.3.2 



Payload I...n (As Required) 

300 

3. 2. 3. 2 

13 

1.3.3 



Reentry Vehicle 

450 

3. 2. 3. 2 

14 

1.3.4 



Orbit I njcctor/Dispcnscr 

225 

3. 2. 3. 3 

15 

1.3.5 



Integration. Assembly, Test and 
Checkout 

75 

3. 2. 3.4 

15 

1.4 


Ground Command, Control, Communications 
and Mission Equipment 

60 

3.2.4 

16 

1.4.1 



Sensor I ... n (As Required) 

20 

3.2.4. 1 

17 

1.4.2 



Telemetry, Tracking and Control 

30 

3. 2.4.2 

18 

1.4.3 



External Communications 

io 

3. 2.4. 3 

19 

1.5 


Flight Support Operations and Services 

50 

4.1.1 

20 

1.5.1 



1 Mate/Checkout/Launch 

30 

4. 1.1.1 

21 

1.5.2 



1 Mission Control 

20 

4.1. 1.2 

22 

1.6 


Systems Engineering/Program Management 

lOO 

5.1.1 

23 

1.7 


System Test and Evaluation 

90 

6.1.1 

24 

1.1.7 


1 Mock-ups 

50 

6. 1.1.1 

25 

1.7.2 


|Test and Evaluation Support 

40 

6. 1.1. 2 

26 

1.8 


Training 

50 

7.1.1 

27 

1.9 


Data 

60 

8.1.1 

28 

1.9.1 



Technical Publications 

IO 

8. 1.1.1 

29 

1.9.2 



Support Data 

20 

8. 1.1. 2 

30 

1.9.3 



Data Depository 

30 

8. 1.1.3 

31 

1.10 


Initial Spares and Repair Parts 

25 

9.1.1 

32 


30 


Basis Of Estimate (BOE) 




CALCULATIONS / ADJUSTMENTS: (show adjustments to data and explain reason for adjustments) 


3 



31 


Time-Phased BCWS 






5 



Schedule 



1^9 Start | & Qfi JH I J Inbox - Microsoft Outlook | [ ^ Microsoft Project - Vie... [EH Microsoft PowerPoint - [P... | U 9:48 AM 
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Work/Planninq Package Summary 



Earned Value Method 

BAC ($K) 

% BAC 

Milestone Weights 

956 

30% 

0/100 

637 

20% 

50/50 

478 

15% 

Apportioned Effort 

159 

5% 

Level of Effort 

478 

15% 

Planning Packages 

478 

15% 

Total 

3185 

100% 


Subcontractor/Other Govt. Activities 



WBS 

Subcontractor/Govt. Activity 

Type 

Cost ($K) 

1100 

Marshall Space Flight Center (MSFC) 


50 

2100 

United Space Alliance (USA) 

CPFF 

200 

3100 

Boeing 

CPIF 

550 

5100 

Lockheed Martin 

CPIF 

30 

5200 

Johnson Space Center (JSC) 


20 

7100 

ASRC 

FFP 

40 

7200 

Dynacs 

FPIF 

40 

8000 

Goddard Space Flight Center 


50 

Total Subcontracts/Govemment Activities 


980 
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Top Risk List & Risk Matrix (Example) 

Program/Project Name from date last review to date current revfeyp 
Risk Matrix 



1 2 3 4 5 

Consequences 


Icriticalitv 

LX C Trend 

Approach 

Med 

E 

Decreasing (Improving) 
Increasing (Worsening) 

M - Mitigate 
W - Watch 

Low 


Unchanged 

A - Accept 


New 

New Since Last Period 

R - Research 


LXC Trend 

Rank 

Risk ID 

Approach 

Risk Title 

* 

1 

Engine-01 

M 

Thermal Vacuum/ Acoustic 
Test 

* 

2 

Engine-06 

W 

On-Orbit Propellant Transfer 


3 

Engine-03 

M 

Aggressive Schedule 

t 

4 

Engine-08 

M 

Government Furnished 
Property (GFP) 

New 

5 

Engine-05 

M 

Hot Fire Test Schedule Slip 

* 

6 

Engine-01 

M 

Design Launch Weight 
Exceeds the Shuttle Capacity 

* 

7 

Engine-04 

R 

Returnability Requirement 

t 

8 

Engine-09 

W 

12 Year Life Certification 

New 

9 

Engine-07 

M 

Reboos Engine/Thruster Lives 

New 

10 

Engine-15 

M 

Unachievable Thruster 
Technology 


9 


Executive Summary 

Cost (Funding) 

0 


Cost (EAC) 

Y 


Schedule 

G 


Technical Performance 

0 


Risk | 

0 


Management Issues: | 

Contractor Transition 
Installation Process 
FR Upgrade 

0 

10 
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Quantified Risk 


Risk Title 

Consequence 

Likelihood 

Pocc 

Cost ($K) 
Impact 

Schedule 

Impact 

Expected 
Value ($) 

Thermal Vacuum/Acoustic 
Test 

4 

95% 

300 

13 wks 

285 

On-Orbit Propellant 
Transfer 

4 

90% 

150 

1 5 wks 

135 

Aggressive Schedule 

5 

80% 

50 

1 5 wks 

40 

Government Furnished 
Property (GFP) 

4 

85% 

75 

1 2 wks 

63.75 

Flot Fire Test Schedule 
Slip 

4 

90% 

125 

1 4 wks 

1 12.5 

Design Launch Weight 
Exceeds the Shuttle 
Capacity 

4 

80% 

200 

1 6 wks 

160 

Returnability Requirement 

4 

60% 

250 

1 5 wks 

150 

12 Year Life Certification 

4 

60% 

225 

1 2 wks 

135 

Reboos Engine/Thruster 
Lives 

3 

80% 

150 

7 wks 

120 

Unachievable Thruster 
Technology 

4 

40% 

175 

1 4 wks 

70 

Total Cost of Risks 
(Cumulative $K) 



1700 



Total Cost of Risks 
Expected ($K) 





1271 .25 

Reserve ($K) 





3000 
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APPENDIX C PROPOSED IBR Language 


Integrated Baseline Review (IBR): The contractor shall present its performance measurement 
baseline plan to the Government within six months after contract award, and subsequently, when 
required, following major changes to the baseline. The Government will verify during the IBR, 
and follow-on IBRs when required that the contractor has established and maintains a reliable 
performance measurement baseline. The contractor will ensure that the baseline includes the 
entire contract technical scope of work consistent with contract schedule requirements, and has 
adequate resources assigned. The contractor will assure the Government that effective earned 
value methods are used to accurately status contract cost, schedule, and technical performance. 
The contractor will perform a self-assessment of the cost and schedule risk for the IBR. The IBR 
will be used to achieve a mutual understanding of the baseline plan, cost and schedule risk, and 
the underlying management processes used for planning and controlling the project. 
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APPENDIX D 


RESPONSIBILITY ASSIGNMENT MATRIX 


FLIGHT DEVELOPMENT PROJECT 

CAM 

BAC ($K) 

1 

Flight Development Project 


72300 

1.1 


Project Management 

CAM 1 

5000 

1.1.1 






1.1.2 






1.2 


Systems Engineering 

CAM 2 

3000 

1.2.1 






1.2.2 






1.3 


Science/Technology 

CAM 3 

2000 

1.3.1 






1.3.2 






1.4 


Safety and Mission Assurance 

CAM 4 

500 

1.4.1 






1.4.2 






1.5 


Payload (s) 


5000 

1.5.1 




CAM 5 

1000 

1.5.2 




CAM 6 

4000 

1.6 


Aeronautical and Spacecraft Systems 


12000 

1.6.1 




CAM 7 

2000 

1.6.2 




CAM 8 

4000 

1.6.3 




CAM 9 

6000 

1.7 


Ground System(s) 


18000 

1.7.1 




CAM 10 

2000 

1.7.2 




CAM 11 

5000 

1.7.3 




CAM 12 

3000 

1.7.4 




CAM 13 

8000 

1.8 


System Integration and Testing 


5000 

1.8.1 




CAM 14 

2000 

1.8.2 




CAM 15 

3000 

1.9 


Launch Vehic 

le/Services 


12000 

1.9.1 




CAM 16 

4000 

1.9.2 




CAM 17 

3000 

1.9.3 




CAM 18 

5000 

1.10 


Mission Operations 


9000 

1.10.1 



CAM 19 

2000 

1.10.2 



CAM 20 

7000 

1.11 


Education and Public Outreach 

CAM 21 

800 
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APPENDIX E 


IBR NOTIFICATION LETTER TEMPLATE 



<Project Name> 

Notification of Integrated Baseline Review 

Supplier: < Supplier Name> 

Contract Number: <Contract #XXXXXXX> 

Date: <Month Day, Year> 
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To: <Supplier Contract Officer Name, Organization> 


SUBJECT: Notification of Integrated Baseline Review (IBR) and Request for Documentation for 

<Subproject Name or Contract #>. 

It is the intent of the <Project Name> Project Office and the Organization or Contractor to use Earned 
Value as a tool in the management of <Subproject Name or Contract #>. In order to support this effort, 
a joint IBR will be conducted in accordance with this contract and current NASA requirements on <Date 
of on-site review>. Detailed scope of the IBR, including specific Control Accounts, will be determined at a 
later date. 

The purpose of the review is to achieve a mutual understanding of the Performance Measurement 
Baseline and its relationship to the Earned Value Management systems and processes. The objectives 
are to gain insight into cost and schedule risk associated with the contracted effort and to establish 
confidence in the project's operating plans. This will be accomplished by jointly evaluating, through 
discussions with your Control Account Managers, the Performance Measurement Baseline to ensure it 
captures the entire technical scope, is consistent with contract schedule requirements and has an 
adequate resource plan. Discussions will focus on the scope of work, work authorization, scheduling, 
resource allocation and time phasing, and earned value techniques used to manage the work. 

Enclosed is a list of documentation requirements that need to be submitted electronically for 
government review no later than <Date>. Forward all electronic submissions to <IBR Coordinator Name, 
Email>. The IBR team requires non-escort badges, a working area with tables and access to printers and 
telephones. 

The IBR kick-off meeting is scheduled for <Date>. Meeting details will be sent separately. The IBR joint 
team training is scheduled for <Date>, Organization or Contractor personnel supporting or 
participating in the IBR are invited. The location is to be determined and will be provided prior to 
training. 

<IBR Team Lead Name> will be the IBR Team Leader for this review. Questions concerning this 
notification or the IBR may be directed to <1 BR Team Lead Name, Phone, Email> or <1 BR Coordinator 
Name, Phone, Email>. 


<PM NAME> 

Project Manager 
<Project Name> Project 

Attachment 1: Documentation and Data Request 
Attachment 2: Review Team On-site Requirements 

Cc: Government COTR 
Review Team Lead 
Review Team Members 
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Documentation and Data Request 


• Work Breakdown Structure (WBS) - to the Control Account level 

• WBS Dictionary 

• SOW with WBS cross-reference index 

• Integrated Master Plan (IMP), if available, or key project milestones 

• Project Risk Register 

• Risk Management Plan 

• Organizational Breakdown Structure (OBS) - to the CAM level 

• Subcontractor listing by WBS element and value of subcontracts, if applicable 

• Contractor/Subcontractor EVMS flow-down requirements 

• Responsibility Assignment Matrix (RAM) showing dollars allocated by Control Account 

• EVM System Description and related business process descriptions (e.g. Risk Management) 

• Initial Control Account Plans (CAP), including Basis of Estimates (BOE), Assumptions, and Risks 

• Latest Estimate at Completion (EAC) and supporting documentation 

• Work Authorization Documents (all levels) 

• Budget Change Requests (BCR) approved thus far, if applicable. 

• Baseline Control Logs (Management Reserve, Undistributed Budget, Contract Budget Base) 

• Integrated Master Schedule (IMS) with linkages to Integrated Master Plan (IMP) 

• Supporting schedules (Intermediate, and Detailed, as applicable) 

• All Contract Performance Reports (CPR) to date 

• Financial Reports such as 533M/Q or equivalent 

• Performance measurement reports used for internal management purposes 

• Control Account/Work Package summary showing: 

o Number of work packages and total value by type of EV method 
o Longest CA, shortest CA, mean and median duration, total value of account 
o Largest CA, smallest CA, mean and median values- Number of Planning packages and 
total Planning Package budget 

• All contract changes to date 

Instruction TO BE DELETED: 

(The above list is intended to be a comprehensive example of the final list. This list should be tailored 
based on information required for preparation and insight. The supplier may have the data described 
above in different formats or naming conventions. Only request data that has not already been received. 
Documentation should be requested to be received NO LATER THAN 2 - 4 weeks prior to the IBR.) 
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Review Team On-site Requirements 


A certain amount of administration is required to ensure the smooth IBR. This effort will benefit all 
stakeholders. The project is requested to arrange the following for our Review Team's on-site presence 
during the review: 

1. All Review Team members are cleared for any security requirements, and that arrangements are 
in place for them to be collected and escorted in necessary while on site. 

2. Provide adequate working area and collaborative work space. 

3. Schedules for discussions with CAMs and other project management staff and ensure all 
personnel are available. 

4. Ensure that rooms for CAM discussion are available. 

5. Administrative support as required. 
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APPENDIX F 


IBR GUIDANCE PACKAGE 


INTEGRATED BASELINE REVIEW 
GUIDANCE PACKAGE 


< PROJECT LOGO> 


<PROJECT NAME> 


<SUPPLIER NAME> 
<SUPPLIER ADDRESS> 
<IBR DATES> 
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PURPOSE 


The purpose of the IBR is to achieve a mutual understanding of the baseline plan and its 
relationship to the underlying Earned Value Management (EVM) systems and processes 
that will operate during the life cycle of the project. The objectives are to gain insight 
into cost, schedule, technical, resource, and management process risk areas, as well as 
develop confidence in the project's operating plans. This will be accomplished by 
evaluating the performance measurement baseline to ensure it captures the entire 
technical scope of work, is consistent with schedule requirements, has adequate 
resources assigned, and has sound management processes. 


PROJECT BACKGROUND 

<Short description of the project and the supplier's role> 
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REVIEW METHODOLOGY 


The IBR process will include the activities listed below. 

a. A review of the documentation that establishes the current and baseline plan 

will occur prior to and during the IBR. This will include technical scope, cost 
estimate to complete (ETC's), basis of estimates, budgets, resource plans, 
schedules, and earned value methods. Concern area reports will be 
generated and submitted as a result of pre-IBR and IBR reviews. 

b. IBR training to familiarize the review team with the IBR process, purpose, and 

documentation. 

c. Discussions with selected managers to verify the adequacy and risk related to 

work authorizations, budgets, ETC's, current and baseline schedules. 

e. Sub-team evaluations, risk assessments, and preparation of concern area 
reports required. Team meetings to discuss results of the control account 
manager discussions. 

f. An out briefing by the Team Leader covering the results and findings of the 
review. The Sub-team Leaders will also be available at the out brief for 
questions and comments. 
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SAMPLE REVIEW AGENDA 


Day 1 

8:00- 10:00 IBR In-brief, Administrative Details 
10:00- 11:30 Documentation Review 
11:30-12:30 Lunch 
12:30- 2:30 Discussion Period 1 

2:30- 3:30 Period 1 Wrap up 

3:30- 5:00 Sub-Team Out-Briefs 


Day 2 

8:30-10:30 

10:30-11:30 

11:30-12:30 

12:30-2:30 

2:30-3:30 

3:30-5:00 

Day 3 

8:00 - 10:00 
10:00-10:30 
10:30-11:30 
11:30-12:30 
12:30-3:00 


Discussion Period 2 
Period 2 Wrap up 
Lunch 

Discussion Period 3 
Period 3 Wrap up 
Sub-Team Out-Briefs 

Discussion Period 4 
Period 4 Wrap up 
Out-brief Preparation 
Lunch 

IBR Out-Brief & Closing Statements 
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SAMPLE IBR QUESTIONS 

(USE AS GUIDELINE FOR DISCUSSION PREPARATION) 


The list of sample questions below should serve as a guide to the IBR Team, and 
does not represent a comprehensive list. It is not the intent for the IBR Team to ask each 
of these questions during the CAM discussions. The IBR Team can review the 
questions below and select the areas relevant to each CAM discussion. 


ORGANIZATION 

What is the manager's scope? 


The manager should be able to refer to a Statement of Work (SOW) paragraph, a 
Contract Work Breakdown Structure (CWBS), or WBS narrative, and a Work 
Authorization Document. 

How many people work for you and what do they do? 

How do they report to you (how do you know the performance status of their work)? 
How did the manager plan the work into control accounts? 

The SOW defines the effort. The WBS or CWBS provides specifics, such as work 
definition. The work authorization and change documentation should show 
information such as the dollars/ hours, period of performance, and description of 
the scope of work and any changes. 

How did the manager ensure that all elements of the scope are planned? 

The manager should be able to show the scope of work broken down into 
work packages, planning packages, or summary level planning packages and 
the budgets and ETC's associated with each. The sum of the work packages 
and planning packages should equal the control account budget. The actual 
costs plus the ETC's should equal the Estimate at Completion. 

How did the manager obtain the resources for assigned work? 

Baseline resources should be identified in the work authorization document 
and changes in scope, cost or schedule requirements should be reflected in 
change request documentation. 

What process did the manager use to develop the resources required to accomplish the 
current plan and how does this differ from the original plan? 
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Review the basis of estimate for reasonableness. 

Does the manager believe that the budget or ETC is sufficient to perform the 
work? Ask the Manager to describe the resource requirement development 
process. 

What is the current EAC? 

How was the EAC developed? 

Who reviews updates to the EAC? 

Does the EAC require program manager approval? 

How does the EAC compare to the BAC? 

Note: The cost estimate to complete (ETC) should be reviewed monthly by the CAM. 

Elicit a range of possibilities (low and high) that represents as clearly as possible the 
complete judgment of the CAM as follows: 

Ask the CAM the basis of estimate (i.e., results from previous projects, etc.). 
Does the estimate consider past performance and does the EAC reflect the 
current cost performance trend? 

Identify risks/ opportunities that are included/ not included in the baseline. 

What are the major risks or challenges remaining to accomplish the CAM's or 
subcontractor's responsibilities? 

Ask the CAM to describe why it is a risk or opportunity. 

Exchange ideas about risks or opportunities. 

Establish the likelihood of the risk/ opportunity event. 

Ask the CAM to explain the risk mitigation plan emphasizing risk mitigation 
milestones and associated risk performance measurement. 

Determine the impact (cost/ schedule) for medium and high risks. 

Ask the CAM to consider extreme values for his effort (optimistic/ pessimistic). 

Document results on the Risk Assessment Form 
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AUTHORIZATION 


How are you authorized to begin work? (Provide an example of work authorization 
documentation.) 

Show me your work authorization document(s), which define the work you must 
accomplish and relate these requirements to the work remaining within your 
team/WBS element at the time the cost to complete, was analyzed/ developed. 


BUDGET 

What role did you play in formulating the budget? 

How did you arrive at your budget figures? Do you have the backup or worksheets 
from which you arrived at your estimates? 

Was there a negotiation process for your budgets? Is your budget adequate? 

How were you advised of budget? Of tasks? Of schedule? Of changes? 

CONTROL ACCOUNT 

How many control accounts are you responsible for and what is the total dollar value of 
your accounts? May we see a control account plan? 

How are your budgets time-phased, and is this reflected in your control account plan? 

How do you status your accounts? How does the performance status of your accounts 
get into the system? 

Do you have any LOE accounts? Please describe the tasks of these accounts. 

Do you have any control accounts that contain a mixture of LOE and discrete effort? 
What is the highest percentage of LOE within an account that also contains discrete 
effort? 

How do you open a control account? 

How do you close a control account? 

How can you tell when a control account is opened or closed? 

What reports do you receive that give you cost and schedule progress of your control 
accounts? 
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WORK PACKAGE 


What percent of your work is measured or discrete effort? What percent is Level of 
Effort (LOE)? 

How does your work package relate to the CWBS or WBS? Please discuss with actual 
examples. 

How are your work package activities related to the Integrated Master Schedule or 
underlying intermediate supporting schedules? Actual examples will support this 
discussion. 

How was the budget time-phased for each work package, i.e., what was the basis for 
the spread? Is the time-phased budget related to planned activities of the work 
package? 

For the example control account, what is your total budget amount? Of this total 
budget amount, how much is distributed to work packages and how much is retained 
in planning packages? Do you have an undistributed budget or management reserve 
account? 

Do you use interim milestones on any of your work packages to measure BCWP? 

How do you define a work package? What is the difference between a work package 
and a planning package? 

How many work packages do you have responsibility for? 

What options does your management system provide for taking BCWP? 

Do your control account plans indicate the method used in taking BCWP? 

How do you know when a work package is opened or closed? 

Have you ever opened work packages earlier than the scheduled start date? If so, how 
is this accomplished? 

Who prepares the budgets for your work packages? 

Demonstrate how you earn BCWP in the same way that BCWS was planned? 

Can you provide examples of how you measure BCWP or earned value for work-in- 
process? 

Does anyone review labor hours charged to your work packages? 

Do you ever have mischarges to your work packages? How are these corrected? 
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PLANNING PACKAGE 


What is the procedure and time frame for discretely developing work packages from 
the planning packages? 

Are your planning packages time-phased? 

SCHEDULE 

What are your schedule responsibilities? 

What schedule milestones did the manager use in planning the control account(s)? Ask 
the Manager to show the team the schedule milestones used in the planning of the 
control accounts. How does the current schedule compare with the baseline schedule? 

The manager should discuss: 

Relationships of work packages to milestones. 

Schedule interfaces and constraints. 

Resource levels to support schedule milestones. 

Relationships to other organizations. 

Schedule impacts related to other work/ organizations. 

Level of Effort tasks that support the schedule. 

How did the manager time-phase the work to achieve the schedule? All work should 
be logically planned in compliance with the SOW and schedule. 

Has the manager considered risks in developing the plan? 

Has the manager adequately planned and time-phased resources to meet the plan? 


Do you directly support any major master or intermediate schedule milestones? 

Do you have detailed schedules below the work package? 

How do detailed schedules below the work package support the work package 
schedules? 

How are you informed by other organizations of changes in their output that may affect 
your control accounts schedules? (Horizontal Trace) 

Demonstrate that the progress reflected on the master project schedule or underlying 
intermediate schedules correlates to the relative progress reflected in the EVM system. 
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CHANGE CONTROL 


Have you had retroactive changes and/ or replanning efforts to the budget baseline? 

Have you had any changes to your accounts? (Provide example of how these are 
handled.) 

Are budget transfers between your accounts and management reserve and 
undistributed budget traceable? How? 

Do you have any work originally planned for in-house that was off-loaded? How was 
this accomplished? 

For off-loaded work, was the budget transferred directly, returned to management 
reserve, or to undistributed budget? 

EARNED VALUE 

Is progress toward accomplishing identified and planned activities used to determine 
earned value? If yes, describe the process. If no, how is earned value assessed? 

What type(s) of earned value measurement indicators have been assigned by the 
manager? 


Is the earned value method chosen appropriate for the type of work being 
performed? 

Does the method chosen objectively measure performance? 

Does the earned value assessment correlate with technical achievement? 

What methods and tools does the manager use in administering the plan? 

Some examples are weekly or monthly earned value reports; master, 
intermediate, and detail schedules; periodic meetings; independent assessments 
of technical progress, etc. 

Determine how changes are incorporated. Evaluate the effect of changes on 
performance measurement information. Assess whether changes are done in 
accordance with the EVM system description or documented management processes. 

What formal training have you had in EVM? 

Do you feel you have had adequate training or do you need more? 
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ESTIMATE AT COMPLETION (EAC) / COST-TO-COMPLETE (CTC) 

What does Estimate at Completion (EAC) mean to you? How do you arrive at an EAC? 
How often is your EAC reviewed and revised? 

What guidance or instructions did you receive from management in order to develop 
your EAC? 

If written instructions were provided, what were these and who authored them? 

Define the work remaining within your WBS element at the time the cost to complete 
was analyzed/ developed. Identify effort to be performed by major subcontractors. 

How did you determine the effort or resource amounts required to complete the 
remaining work? 

■ Outline the steps you took to arrive at your estimate. 

■ What project/ performance risks have been considered in your estimate? 

■ What performance level was assumed and why? 

■ How does the projected performance level compare to your experienced 
level of performance? 

■ How are EAC's calculated for material? 

Demonstrate that your EAC is segregated by labor, material, and other direct charge 
categories. 

What current and future events and performance factors have been included in your 
current cost to complete? Examples: task changes, make-buy decisions, performance 
factors, etc. 

Describe and demonstrate how you projected the cost to complete over the time 
remaining. 

Discuss your management's involvement in developing the estimate of the cost 
remaining to complete your program tasks. 

SUBCONTRACTOR/EXTERNAL WORK 

Are you responsible for any subcontracts? How do you monitor performance on these? 
How do you take BCWP? 
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How are subcontracts managed? Ask the Subcontracts Manager to describe the process 
for managing subcontractor earned value. 

What subcontracts are your responsibility? What criteria determine whether a 
subcontract or a Purchase Order is used? What types of subcontracts exist or plan to be 
negotiated (fixed price vs. cost plus)? 

What are the major challenges or risks to the subcontractor in accomplishing project 
responsibilities? 

Are these items tracked by the Project Management Office or Functional Manager in a 
risk register or plan? 

What subcontractor technical, schedule and cost reports are required to be submitted to 
you or your team? 

What is your total budget (for each subcontract and the corresponding control 
accounts)? How is profit or fee included in your budget? 

How was the budget established? Does it reflect an achievable value for the resources 
to fully accomplish the control account scope of effort? 

What rationale was used to time phase the budget into planning packages, tasks, work 
packages or summary activities? 

Are the time phased budget resources consistent with your integrated master schedule? 
Show the trace from your control account to intermediate or master schedules. 

When are you required to detail plan planning packages or summary activities? What 
schedule document or system is used to develop detail planning for your control 
account? 

How do you know that the work within your control accounts to be performed by 
subcontractor has been properly planned? 

How do you check the status and performance of work on your control account by a 
subcontractor? 

How are actual costs recorded against your control account? 

What techniques are available for determining Earned Value? Explain the application 
of each technique. 

How and when is risk assessment or risk management plan updated for 
technical/ schedule/ cost risk items affecting your control account? 
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How and when is the actual and forecast schedule update provided for your control 
account effort? 

Are variance analysis thresholds or requirements established for reporting technical, 
schedule or cost variances to planned goals established for your control accounts? Do 
you informally/ formally report the cause of variance, impact or corrective action for 
these variances? 

What is your current Estimated Cost at Completion (EAC)? How often is it updated? 
Does your EAC reflect current cost performance trend? 

What document authorizes you to begin work on a subcontract? 

For these selected work packages, what specific outputs, products, or objectives are to 
be accomplished? 

What is specifically needed by you from other control account managers to generate 
subcontractor outputs or products? How do you monitor its progress? 

Who specifically needs the subcontractor outputs or products to perform their program 
functions? How do you status others on the progress of your outputs to them? 

Specifically, what technical items are currently producing the greatest risk to achieving 
technical, schedule or cost goals? Are these items reviewed as part of a risk assessment, 
management plan or other reporting tool to your boss or the project management 
office? 

How do you determine whether the reported cost variance is due to subcontractor 
effort or a company overhead rate? 

How are material budgets planned? 

How do you track material prior to delivery? 

How do you track material when deliveries are late? 

When is BCWP or earned value taken on material? 

How much BCWP is earned when material is withdrawn from inventory or received? 

ANALYSIS 

Do you have any variance thresholds of your control accounts? 

What are the variance thresholds of your control accounts? 
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How do you know when you have exceeded a threshold? 

How do rate changes affect your control accounts? 

Who is responsible for rate variance analysis? 

Will an account accept BCWP or ACWP if there is no BCWS? 

How do you know when you must prepare a variance report? 

Do you have samples of any variance analysis reports? (Do these show a statement of 
problem, the variance, cause, impact and proposed corrective action?) 

Who receives your variance reports? What action is taken on the reports? 

Which reports do you use most frequently? Why? 


OTHER 

How are you reporting labor, material, and other direct costs? 

Has your effort been impacted by any directed or contractual change? When did you 
receive authorization to proceed with the change and how did you incorporate the 
change in your plan (schedule and budget time phasing)? 

What changes have been made to the control account planning (technical definition of 
scope, schedule, budget resources, ETC's)? 

■ What documents are involved in a change to the control accounts' scope of 
work, schedule, budget, or ETC? 

■ Did the CAM re-phase or replan work? In process work? Completed work? 
Unopened work packages? Make current period or retroactive changes? 

■ Did the CAM transfer budget between control accounts? 

■ How have contract changes or other changes been incorporated into the 
control account? 

■ If one of the control accounts had an unfavorable cost or schedule variance 
did the CAM replan or request management reserve to reduce or eliminate 
the variance? 
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DISCUSSION GUIDELINES 


LOCATION OF DISCUSSION 

Conference Rooms will be provided by the supplier for the on-site review. 


STRUCTURING THE DISCUSSION 

■ Have an objective. What is the purpose for speaking with this particular 
manager? What do you expect to gain from the discussion? 

■ What questions will you ask to achieve the objective? 

■ Prepare a tentative list of basic questions to serve as a framework for the 
discussion. This will open the way for spontaneous in-depth conversation and 
follow-up questions. 

■ Designate a person to take notes 


THE DISCUSSION PROCESS 

Below is a list of suggested techniques for all team members to consider before and 
during the CAM discussions. 

■ Prior to the discussions, the IBR Team members should be familiar with areas 
previously identified for discussion. Review the documentation thoroughly. 

■ Introduce yourself and identify the organization you represent. You may also 
wish to indicate your team affiliation in the review. 

■ Be well prepared and maintain a tempo that keeps the discussion moving 
along toward satisfying your objective. Be friendly, but avoid long 
conversations extraneous to the discussion. 

■ Take notes. The discussion leader (normally the Sub-team Technical Lead) or 
an accompanying team member should take notes during the discussion. 

■ Request copies of documents only if necessary to accomplish the objective of 
the discussion. If documentation is not readily available, complete a 
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Documentation Request Form. See Appendix F for a sample Documentation 
Request Form. 

■ Watch the time. Discussions are normally scheduled for two hours in length. 
Should additional time be required to complete the discussion, coordinate 
with the Team Leader 

■ If disagreements arise which cannot be resolved, the team member should 
write a description of the disagreement in a Concern Area Report and submit 
it to the Team Leader for disposition. The Team Leader will handle any 
continuing discussion with the Project Team. 


AFTER THE DISCUSSION 

Complete the written discussion assessment form directly after the discussion. Use this 
as a reference during the daily Government team meetings. Submit the forms and all 
concern area reports to the IBR Coordinator. 

Be sure to follow up on any outstanding issues or questions, especially if you address 
them in your discussion assessment forms, or if you have committed to obtaining 
additional information in support of your assessments. Do not leave any loose ends. 


TEAM ASSESSMENTS: 

The following provides additional guidance for assessing the adequacy of the scope, 
schedules, resource plans, and earned value methods. 

a. Technical Scope: Evaluate the technical content described by the CAM and 

compare that to the definitions provided in the CWBS or WBS Dictionary, 
control accounts, and work packages. Ensure that the work scope described 
is consistent with the Statement of Work and specifications and that it is clear, 
sufficiently detailed, well understood, and complete. Ask the CAM if there is 
any planned effort that is not included or fully identified in the 
documentation. 

b. Schedules: Examine the detailed, intermediate, and master schedules. The 

scheduled milestones and activities reflected in the work packages and 
control account plans should be consistent with intermediate schedules and 
project milestones. The sequence of planned activities/ events should be 
logical. Time frames allocated to accomplish these schedules should be 
reasonable. To the extent practicable, schedules should identify the 
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significant task interdependencies needed to meet the requirements of the 
project. Significant decision points, constraints, and interfaces should be 
identified as key milestones. Ask the CAM to identify any known risks or 
barriers in meeting the schedules. 

c. Resource Plan: Determine the adequacy of the amount and time-phasing of 

hours, materials, and other required resources planned to accomplish the 
work identified in control accounts, work packages, and planning packages. 
The basis of estimate should be reasonable. Determine whether the supplier 
has or anticipates potential difficulties in obtaining or using the required 
resources to accomplish the work. Ask the CAM to identify any known risks 
in meeting the plan. 

d. Earned Value Methods: Be familiar with the supplier's earned value methods. 

Ask the CAM why a particular type of earned value method was chosen. 
Determine whether the supplier is using the best method for the identified 
type of work. There should be a meaningful correlation between technical 
achievement and cost and schedule control. Ask the CAM how he or she 
ensures that measurable earned value assessments correlate with technical 
achievement. Ensure that earned value methods and assessments are as 
objective as possible, that the assessments represent actual technical progress, 
and that a reasonable quantitative approach exists to assess technical status. 
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DOCUMENTATION GUIDELINES 


Each Sub-Team Leader is responsible for documenting the sub team's 
assessments and findings. The Team Leader uses the documentation to support overall 
team assessments and required corrective actions. 

The following forms have been developed to facilitate review documentation. 

a. Discussion Assessment Form: The Team/Sub-team should complete one 
of these forms after each discussion. The Team/Sub-team Technical Leader is 
responsible for reviewing this form and submitting it to the Team Leader. 
Concerns noted should be documented in the Concern Area Report (CAR). Keep 
in mind that the Discussion Assessment Lorm can be tailored based on risks 
associated with the project. 

b. Concern Area Report (CAR) : A CAR should be completed for each 
concern noted. Generally, concerns noted are items that require follow-up 
action. Information on this form should be clear and as specific as possible 
because it will be provided to the supplier to obtain a response and resolution. 
Spell out abbreviations. These forms should be submitted immediately to the 
Team Leader, and as soon as practical to the supplier. Responses should be 
provided to the team as soon as possible. Again, the Team/Sub-team Leader is 
responsible for reviewing this form, and submitting it to the Team Leader. All 
forms will be compiled and logged. 

c. Documentation Request Form: During your discussions, you may find 
that you require additional documentation to gain a better understanding of the 
issue in question. Use the form to obtain required documentation. Submit the 
completed form to the Team Leader. All forms will be tracked in order to reduce 
redundancies and ensure receipt of all the requested material. 


d. Risk Assessment Form : A risk assessment form should be completed after 
each discussion. Each risk and opportunity identified during the CAM 
Discussion is assigned a probability of occurrence. A potential cost and schedule 
impact for each risk and opportunity should be estimated, and a determination 
made as to whether this risk has been accounted for in the baseline. Each risk is 
classified as cost, schedule, technical, resource, or management process and 
assigned a rating based on risk evaluation criteria established by the NASA PM 
prior to the IBR. The ultimate goal of the Risk Assessment area of the IBR is an 
updated Estimate At Complete (EAC) or Life Cycle Cost Estimate (LCCE) which 
incorporates quantified risks. The Team/Sub-team Leader is responsible for 
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reviewing this form and submitting it to the Team Leader. Risks identified at the 
IBR should be incorporated into the project's existing risk management plan. 

This is just a sample. If your program or project has a specific risk management 
process, be sure to use that methodology. For Constellation Program IBR's, see 
Appendix P of the CxP Candidate Risk Assessment Form and the CxP IBR 
Management Guide for risk evaluation criteria and guidance. 

e. General Assessment Form : The form is used to document observations 
for those things that are "not CAR worthy", but might help EVM efficiencies 
(such as automatic links between IMS and EVM engine), as well as potential 
problem areas in the future based on previous IBR lessons learned. It is also 
used to document "best practices" for those things that improve the EVM 
process. 
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IBR DISCUSSION ASSESSMENT FORM 


LOG# Team Date: 

Manager: Area of Responsibility: 

2. TECHNICAL SCOPE (Statement of Work): 

Is there adequate identification, definition, and flow down? 

Consistent with contract requirements? 

Adequate assignment of responsibility, authority & accountability? 

2. SCHEDULES Period of Performance: 

_____ Realistic planned durations? 

_____ Logical sequence of work planned? 

____ Consistent with intermediate/ master schedule? 

_____ Significant interdependencies, interfaces, & constraints? 

Support contract milestones? 

3. COST AND RESOURCE RISK 

Sound basis of estimate? 

Budget reasonableness (time phasing, levels, mix, type)? 

_____ Budget adequacy (time phasing, levels, mix, type)? 

_____ Resource availability? 

Adequate budget/ etc values assigned? 

Provisions for scrap, rework, retest or repair? 

4. MANAGEMENT PROCESS RISK 

____ Integrated cost/ schedule/ technical planning? 

_____ Baseline change control? 

_____ Accurate and timely management/ performance data? 

_____ Adequate determination and maintenance of EAC's? 

Adequate subcontract management? 

Risk management process documents risk associated with the PMB? 

Appropriate planned earned value methods? 

Objective determination of progress? 

_____ Methods correlate with technical achievement? 

5. Brief Summary of Discussion 

6. Concern Area Report (CAR) prepared? 
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IBR CONCERN AREA REPORT 


WBS / CA Log #. 

Date: 


Submitted by: _________________________ 

Subject of Finding: 

Discussion (Explain root problem and cause. Provide impact assessment. Quantify 
problem and impacts where possible. Provide recommended actions and exit criteria 
for resolution. Attach exhibits if applicable. Provide reference to control account or 
work package number). 


Supplier's response (Address root cause of the problem, impact, 

corrective/ preventative action plan; identify dates and POC. Identify exit criteria for 

corrective action). 


Sub-team leader signature: 
Team leader signature: 
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DOCUMENTATION REQUEST FORM 


Log #: Sub-Team. 

Date: ____________ 

Submitted by: 

1. Responsible Manager: 

Control Account(s): 

Area(s) of Responsibility: 

2. Document Description or Type. 

3. Reason for Request: 

4. Remarks/ Comments: 


Sub-team Leader Signature: 


Team Leader Signature: 
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RISK EVALUATION CRITERIA 


The IBR Discussion Evaluation Summary form and IBR Out Brief need all 5 types 
of risks for an IBR: Risks can generally be categorized into the following five areas: 
technical, schedule, cost, resource, and management processes. The following are brief 
discussions of each of the types of risk. 

Technical Risk - The ability of the project's technical plan to achieve the objectives of 
the scope of work. Technical risk includes the effects of available technology, software 
development capability, design maturity, etc. 


Schedule Risk - The adequacy of the time allocated for performing the defined tasks to 
successfully achieve the project schedule objectives. Schedule risk includes the effects 
on the schedule of the interdependency of scheduled activities to achieve project 
milestones and support the PMs' ability to identify and maintain the critical path. 

Cost Risk - The ability of the PMB to successfully execute the project and attain cost 
objectives, recognizing the relationship between budget, resources, funding, schedule, 
and scope of work. The quality of the estimates affects the cost risk, which includes the 
assumptions used for both estimates and resource allocation on the budgets for work 
items. 

Resource Risk - The availability of personnel, facilities, and equipment, when required, 
to perform the defined tasks needed to execute the program successfully. Resource risk 
includes the effect of external factors such as loss of availability to competing programs 
or unexpected downtime that could preclude or otherwise limit the availability of the 
resources needed to complete planned work. 


Management Processes Risk - The degree to which the management processes provide 
effective and integrated technical/ schedule/ cost planning and baseline change control. 
Management processes risk includes the ability to establish and maintain valid, 
accurate, and timely performance data, including data from subcontractors, for early 
visibility into risks. 

The following paragraphs and one chart explain the criteria for evaluating the five risks 
described above. 
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Sample Technical, Schedule, Cost, Resource, and Management Process Risk - 
Evaluation Criteria: 


© 

© 

jS 

*3 


Project Risk Analysis 


What is the likelihood the risk will happen? 

Lvl 

Likelihood 

Planned approach and processes 

l 

Not likely 

Will effectively avoid or mitigate this 
risk based on standard practices 

2 

Low 

likelihood 

Have usually mitigated this type of risk 
with minimal oversight in similar cases 

3 

Likely 

May mitigate this risk, but workarounds 
will be required 

4 

Highly likely 

Cannot mitigate this risk, but a different 
approach might 

5 

Near 

certainty 

Cannot mitigate this type of risk, no 
known processes or workarounds are 
available 




X3 

O 

O 

£ 

13 
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^^Consequence 




Given the risk is realized, what would be the magnitude of the impact? 

Lvl 

Technical 

Schedule 

Cost 

l 

Minimal or no impact 

Minimal or no impact 

Minimal or no impact 

2 

Minor performance shortfall, same 
approach retained 

Additional activities required, able to 
meet key dates 

Budget increase or unit production 
cost increase <1% 

3 

Moderate performance shortfall, but 
workarounds available 

Minor schedule slip, will miss need 
date 

Budget increase or unit production 
cost increase <5% 

4 

Unacceptable, but workarounds 
available 

Program critical path affected 

Budget increase or unit production 
cost increase <10% 

5 

Unacceptable, no alternatives exist 

Cannot achieve key program 
milestone 

Budget increase or unit production 
cost increase >10% 
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Resource Risk - Evaluation Criteria : 

Excellent (Green) - Has little potential to cause disruption of schedule, 
increased cost, or degradation of performance. Normal supplier effort 
and normal Government monitoring will probably be able to overcome 
difficulties. 

Adequate (Yellow) - Can potentially cause some disruption of schedule, 
increased cost, or degradation of performance. Special supplier emphasis 
and close Government monitoring will probably be able to overcome 
difficulties. 
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Poor (Red) - Likely to cause a significant disruption of schedule, increased 
cost, or degradation of performance. Risk may be unacceptable even with 
supplier or emphasis and close Government monitoring. 


Management Processes Risk - Evaluation Criteria : 

Excellent (Green) - Processes are in place for Baseline Maintenance, Risk 
Management, Scheduling, Estimate At Completion updates. Subcontract 
Management and Managerial Analysis. Earned value methods are 
appropriate, provide objective determination of progress, and correlate 
with technical achievement. These processes are formally documented 
and are being used to manage the program. Few issues have been 
identified with the processes or how they are being applied. Management 
processes will provide timely and accurate performance data. Has little 
potential to cause disruption of schedule, increased cost, or degradation of 
performance. Normal supplier effort and normal Government monitoring 
will probably be able to overcome difficulties. 

Adequate (Yellow) - Most, but not all, processes are in place for Baseline 
Maintenance, Risk Management, Scheduling, Estimate At Completion 
updates. Subcontract Management and Managerial Analysis. Earned 
value methods could be more objective and correlate more closely with 
technical achievement. Some processes are not fully documented. 

Discussions indicate that the Control Account Managers area not correctly 
using the management processes. There are concerns that the 
management processes may hinder timely and accurate performance data. 

Can potentially cause some disruption of schedule, increased cost, or 
degradation of performance. Special supplier emphasis and close 
Government monitoring will probably be able to overcome difficulties. 

Poor (Red) - Few management processes are in place for Baseline Maintenance, 
Risk Management, Scheduling, Estimate At Completion updates. Subcontract 
Management, and Managerial Analysis. Earned value methods are subjective 
and do not correlate with technical achievement. Processes are not documented. 
Discussions indicate that the Control Account Managers are not using the 
management processes There are concerns that the management processes will 
prevent accurate and timely performance data. Likely to cause a significant 
disruption of schedule, increased cost, or degradation of performance. Risk may 
be unacceptable even with supplier emphasis and close Government 

A format for assessment by process area is shown below and may be a helpful 
tool to the team in characterizing and consolidating management process 
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findings. This format can also be used to support the IBR Findings Letter or IBR 
Report. 


Management 

Process 

Area 

Deficiency 



Low 

Moderate 

High 

No 

Deficiencies 

Best 

Practice 

Organization 






Work Authorization 






Scheduling 






Budgeting 






Status/Updates 






Management Analysis 






Change Control 






EAC and Risk Management 






Accounting 






Indirect Management 






Material Management 






Contract/Subcontract 

Management 






Training 












Total 








Low (No CAR) 


Moderate (Could warrant a CAR) 


High (Should have corresponding CAR) 


No Deficiencies 


Best Practice 
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IBR RISK ASSESSMENT 


Sub-Team: Date: 

Control Account: WBS ID: 


Risk/ Opportunity Description Probability 

Cost 

Schedule 

In 

Risk 

Risk 

of 

Impact 

Impact 

Baseline? 

Type 

Rating 

Occurrence 



(Y/N) 


(R/Y/G) 

(H/M/L) 
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GENERAL ASSESSMENT FORM 


General Assessment: Is intended to document observations for those things 
that are “not CAR worthy”, but might help EVM efficiencies (such as 

Date: 

automatic links between IMS and EVM engine), as well as potential problem 
areas in the future based on previous IBR lessons learned. It is also used to 
document “best practices” for those things that improve the EVM process. 

Assessment: 

Technical (Yes or No): 


Cost (Yes or No): 


Schedule (Yes or No): 


Owning Organization: 


Condition Statement: Is a phrase briefly describing current key circumstances, situations, etc. that are “not CAR worthy”, 
but might help EVM efficiencies. 


Action Statement: Is a phrase briefly describing the appropriate actions required. 


Best Practices: Is a phrase briefly describing the “best practices”. 


IBR Team Member Signature: 
Print Name: 


IBR Team/Sub-team Lead Signature: 
Print Name: 


IBR Risk Management Officer Signature: 
Print Name: 
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GLOSSARY OF EVM TERMS 


For the in-house portion of a project, use the NASA financial Management 
Requirements (FMR) document for definitions of Center G&A, corporate G&A, Direct 
Costs, G&A, Indirect Cost, Service Pools, and Service Pool costs 

Actual Cost of Work Performed (ACWP) (or Actual Costs). The costs actually incurred 
and recorded in accomplishing the work performed within a given time period. Actual 
costs include the direct cost plus the related indirect cost such as overhead, G&A, etc. 
allocated to the activity. 

Apportioned Effort . Effort which by itself is not readily divisible into short-span work 
packages but which is related in direct proportion to some other measured effort. 
Authorized Unpriced Work . Any effort for which contractually definitized costs have 
not agreed upon, but for which written authorization has been received. 

Authorized Work . Effort which has been definitized and is on contract, plus that effort 
for which definitized contract costs have not been agreed to but for which written 
authorization has been received. 

Baseline . See "Performance Measurement Baseline." 

Bottoms-Up Cost Estimate . An estimate derived by summing detailed cost estimates 
of the individual work packages and adding estimated level of effort plus appropriate 
indirect cost estimates. Frequently accomplished as an independent analysis by 
Industrial Engineering, Price Analysis and Cost Accounting. 

Budget . A plan of operations for a fiscal period in terms of estimated costs or hours. 
Budget At Completion (BAC) . The sum of all budgets (BCWS) allocated to the project 
or a given Control Account. It is synonymous with the term Performance Measurement 
Baseline. 

Budgeted Cost For Work Performed (BCWP) (or Earned Value). The sum of budgets 
for completed work packages and partially completed work packages, plus the 
appropriate portion of the budgets for level of effort and apportioned effort work 
packages. 
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Budgeted Cost For Work Scheduled (BCWS) (or Planned Value). The sum of the 

budgets for all work packages, planning packages, etc., scheduled to be accomplished 
(including in-process work packages), plus the amount of level of effort and 
apportioned effort scheduled to be accomplished within a given time period. This is the 
value of planned work. 

Budgeting . The process of translating approved resource requirements into a time- 
phased plan for accomplishing work. 

Change Order . A formal revision or change to a contract or project that may impact the 
scope of work, schedule, cost, price, and/ or other requirements. 

Contract Budget Base (CBB) . The negotiated contract cost plus the estimated cost of 
authorized but unpriced work. It includes the PMB and MR. Customer approval is 
generally required to change it. 

Contract Performance Report (CPR) . A contractually required recurring report 
designed to provide information on contract cost performance status, problems, and 
corrective actions (DI-MGMT-81466). 

Contracting Officer's Technical Representative (COTR) . Person responsible for the 
contractor's performance on a given contract. 

Control Account (CA) . A management control point at which budgets (resource plans) 
and actual costs are accumulated and compared to the earned value for management 
control purposes. A control account is a natural measurement point for planning and 
control since it represents the work assigned to one responsible organizational element 
(or integrated product team) for a single Work Breakdown Structure (WBS) element. 
Control Account Manager (CAM) . A manager responsible for task performance of a 
Control Account and for planning and managing the resources authorized to 
accomplish such task. 

Cost-At-Completion (CAC) . Actual direct costs, plus indirect costs, plus the estimate of 
costs (direct and indirect) for authorized work remaining. The CAC is sometimes 
referred to as Estimate-at-Completion (EAC) or Latest Revised Estimate (LRE). 
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Cost Element . Typical elements of cost are: direct labor, direct material, other direct 
costs, and indirect cost (overhead). 

Cost Performance Index (CPI) . A measure of cost efficiency. It compares BCWP to the 
actual cost to perform that work (CPI=BCWP/ ACWP). An index of 1.0 means that we 
are spending exactly what we planned to spend to accomplish the work performed. 
CPI >1.0 means we are under running costs. CPI <1.0 means that we are over running 
costs. 

Cost Plus Award Fee/Fixed Fee/Incentive Fee . Cost reimbursement contract types 
where the fee is based upon: (a) the accomplishment of pre-negotiated goals (Award 
Fee); (b) a negotiated fixed fee amount (Fixed Fee); or (c) a risk sharing ratio within a 
pre-determined range based upon the contractor's ability to meet technical, cost, and/or 
schedule targets (Incentive Fee). 

Cost Reimbursement Contracts . A category of contracts whose use is based upon 
payment by the government to a contractor of allowable cost as prescribed by the 
contract. Normally only the "best efforts" of the contractor are required. The basis for 
payment negotiated may be: (a) cost (no fee); (b) cost sharing; (c) cost-plus-fixed fee; (d) 
cost plus award fee, and/or (e) cost-plus incentive fee. 

Cost-To-Complete (CTC) Forecast . The performing activity's estimate of the cost to 
complete the remaining tasks. Synonymous with Estimate to Complete. 

Cost Variance (CV) . A metric for the cost performance derived from earned value data. 
It is the algebraic difference between the earned value (BCWP) and the actual cost 
incurred (ACWP). A positive value indicates a favorable condition and a negative 
value indicates an unfavorable condition. 

Critical Path . A sequential path of activities that are tied together network logic that 
have the longest overall duration from time now until project completion. 

Critical Subcontractor . A contractor performing a large or complex portion of a 
contract that requires a flow-down of earned value management and reporting (e.g., 
CPR) requirements, and the integration, reviews, acceptance and control of 
subcontractor system and reporting by the prime contractor. Critical subcontractors are 
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designated as a result of customer negotiation or by management or by government 
direction. 

Defense Contract Management Agency (PCM A). The Department of Defense (DoD) 
component that works directly with Defense suppliers to help ensure that DoD, Federal, 
and allied government supplies and services are delivered on time, at projected cost, 
and meet all performance requirements. As the DoD Executive Agent for EVMS, DCMA 
is responsible for ensuring the integrity and application effectiveness of contractor 
EVMS. The NASA Program/ Project contracting officer will normally delegate the 
responsibility for verifying a supplier's initial and continuing compliance with 
ANSI/EIA-748 guidelines to the designated DCMA Administrating Contracting Officer 
(ACO) assigned to a DCMA Contract Management Office (CMO). 

Defense Contract Audit Agency (DC A A) . The DoD organization tasked with 

monitoring a contractor's design and implementation of an acceptable accounting 
system. 

Definitized . A contract, contract amendment, or contract supplemental agreement is 
considered definitized when the final contractual documents are unconditionally 
executed by both parties to the agreement. 

Direct Cost . Any costs that may be identified specifically with a particular cost 
objective. That is, the portion of labor, material or other cost incurred or expended to 
meet specifications for an end product, tool or other related service specifically 
identifiable to the authorized task. It consists of those costs that can be reasonably and 
consistently related directly and finally without distribution through an overhead unit 
or account. 

Direct Labor . That portion of labor expended in the actual design, tooling, testing and 
the physical application of labor (including proofing) to material altering its shape, 
form, nature, or fulfilling a requirement for service. 

Discrete Effort . Tasks which have a specific end product or end result, and which 
through planning: (1) can be specifically defined and assigned a budget for 
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accomplishment; (2) can be scheduled with clearly definable start and completion dates; 
and (3) contain criteria against which performance can be measured. 

Discrete Milestone . A milestone which has a definite, scheduled occurrence in time 
signaling the finish of an activity, such as "release drawing," "pipe inspection 
complete," and/or signaling the start of a new activity. A type of "objective indicator." 
Earned Value (EV1 . The budgeted value for work accomplished. The value of 
completed work expressed in terms of the budget assigned to that work. It is the sum 
of budgets for completed work packages and completed portions of open work 
packages, plus the appropriate portion of the budgets for level of effort and 
apportioned effort. Also known as Budgeted Cost for Work Performed (BCWP). 

Earned Value Management (EVM) . A tool for measuring and assessing project 
performance through the integration of technical scope with schedule and cost objectives 
during the execution of the project. EVM provides quantification of technical progress, 
enabling management to gain insight into project status and project completion costs and 
schedules. Two essential characteristics of successful EVM are EVM system data integrity and 
carefully targeted monthly EVM data analyses (i.e., risky WBS elements). 

Earned Value Management System (EVMS) . The integrated set of policies, processes, 
systems and practices that meet an organization's implementation of ANSI/EIA-748. 
An integrated management system and its related subsystems that allow for planning 
all work scope to completion; assignment of authority and responsibility at the work 
performance level; integration of the cost, schedule, and technical aspects of the work 
into a detailed baseline plan; objective measurement of progress (earned value) at the 
work performance level; accumulation and assignment of actual costs; analysis of 
variances from plans; summarization and reporting of performance data to higher 
levels of management for action; forecast of achievement of milestones and completion 
of events; forecast of final costs; and disciplined baseline maintenance and 
incorporation of baseline revisions in a timely manner. 
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Earned Value Management System (EVMS) Guidelines . The set of 32 guidelines 
established in EIA-748, which define the parameters within which the integrated cost 
and schedule management system must fit. 

Estimate At Completion (EAC) . A value (expressed in dollars and/or hours) 
developed to represent a realistic project ion of the final cost of a task (or group of tasks) 
when completed. EAC is the sum of direct and indirect costs to date, plus the estimate 
of costs for all authorized remaining work. EAC=Inception to data ACWP+ETC 
Estimate To Complete (ETC) . That portion of the EAC that addresses total expected 
costs for all work remaining. 

Fiscal Year . For the United States government, it is the 12 month period: 1 October 
through 30 September. For industry, it may be any formally selected annual accounting 
period including a calendar year. 

Fixed Price Contracts . A category of contracts based on the establishment of a price to 
accomplish the required work. Types are: (a) firm fixed price; (b) fixed price with 
escalation; (c) fixed price redeterminable, and (d) fixed price with incentive provisions. 
Front Loading . An action to provide adequate or generous budget in the near-term 
budget baseline at the expense of the far-term effort. This practice delays 
acknowledgment of potential overrun conditions, frequently in the expectation of 
recovery through subsequent changes in the statement of work. Front loading often 
results from an inadequate or unrealistic budgets and/or unrealistically optimistic 
planning of far-term effort. 

Functional Organization . An organization or group of organizations with a common 
operational orientation such as Engineering, Manufacturing, (Fabrication, Assembly), 
Tooling, Quality Control, Material, Finance, Contracts, etc. See Organization 
Breakdown Structure (OBS). 

Functional Manager . A line manager or supervisor of a functional organization. 
Funding Profile . A display of project funding requirements normally presented on a 
cumulative basis by months or quarters. 
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Gantt Chart . A horizontal bar chart. A graphic representation used as an aid to 
effective scheduling and control, showing graphically on a time scale when certain 
events are to take place or where deadlines occur. Status is displayed by either filling 
the bar or adding a filled-in status bar below it. 

General & Administrative Expense (G&A) . Expense incurred in the overall Corporate 
and Division offices, i.e., their cost of staff services such as, finance, legal, contract 
administration, sales, marketing and independent research and development (IR&D) 
effort. These expenses are allocated to organizational elements on the basis of total 
direct and indirect cost. 

Giver/Receiver Schedule . The logical relationships and interdependencies in a Critical 
Path Methodology (CPM) based schedule supported by the matching of deliverables 
and users relationships for all products produced by the Project. The Project produces a 
report from the schedule system which demonstrates these relationships for all 
deliverables produced. 

Indirect Budget . The target value established for costs to be incurred by persons 
and/or organizational elements for tasks or expenses that do not have a direct 
relationship to the design, testing and/or production of the end product or specified 
task. 

Indirect Cost . Costs that cannot be specifically or immediately identified to a project, 
but can subsequently be traced or linked to a project and are assigned based on usage or 
consumption. 

Costs that, because of their incurrence for common or joint objectives, are not readily 
subject to treatment as direct costs. 

Indirect Cost Pools . A grouping of indirect costs identified with two or more cost 
objectives but not separately identified with any final cost objective. Such separate 
pools are normally established for indirect costs associated with Engineering, 
Manufacturing, Procurement, and/ or Material, etc. 

Interim Budget . Furnished to departments on an interim basis for authorized tasks for 
which a firm bid or estimate may not yet have been completed and negotiated. It is also 
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used for task transfers between departments or elements of cost pending formal budget 
transfer. Interim budget is discontinued after budget values are formalized. 

Toint Surveillance . Continual observation, by a team of Government and contractor 
representatives, of a supplier's earned value management system to monitor its 
compliance with the EIA-748 Guidelines and its proper use as a management tool. 

Level Of Effort (LOE) . Effort of a general or supportive nature that does not produce 
definite end products. 

Management Reserve (MR) . An amount of the total allocated budget withheld for 
management control purposes rather than designated for the accomplishment of a 
specific task or set of tasks. It is not a part of the Performance Measurement Baseline. 
Material . Property which may be incorporated into or attached to an end item to be 
delivered, or which may be consumed or expended in the performance. It includes, but 
is not limited to, raw and processed material, parts, components, assemblies, fuels and 
lubricants, and small tools and supplies. 

Milestones . Events of particular significance. Finitely defined events that constitute 
the start or completion of a task or occurrence of an objective criterion for 
accomplishment. Milestones should be discretely identifiable; the passage of time alone 
is not sufficient to constitute a milestone. However, milestones should be associated 
with schedule data to document when the milestone is to occur. (See Objective 
Indicator.) 

Negotiated Contract Cost (NCC) . The estimated cost negotiated in a cost-plus-fixed-fee 
contract or the negotiated contract target cost in either a fixed-price incentive contract or 
a cost-plus-incentive-fee (or award fee) contract. 

Network Schedule . A schedule format in which the activities and milestones are 
represented along with the interdependencies between activities. It expresses the logic 
as to how the work scope will be accomplished. Network schedules are the basis for 
critical path analysis, a method for identification and assessment of schedule priorities 
and impacts. 
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Non-Recurring Cost . Expenditures for specific tasks that are expected to occur only 
once, or very infrequently, on a given project. Examples are such costs as those for 
preliminary design effort, qualification testing, initial tooling, testing, planning, etc. 
Objective Indicator . A finite event or accomplishment that can be used to definitively 
establish the degree of completion of a specific task. 

Organizational Breakdown Structure (OBS) . A family-tree breakdown of an 
organization showing the organizational elements involved in performing the work. 
Other Direct Cost (ODC) . A group of accounting elements, other than direct labor and 
material, which can be isolated to specific tasks. ODC includes such items as travel, 
computer time, aviation fuel, etc. 

Overhead . (See Indirect Costs.) 

Overrun . Costs incurred in excess of the allocated budget for a task, group of tasks, 
project, or a contract. For work in process, the overrun is the cost incurred in excess of 
earned value. 

Over Target Baseline . A performance measurement baseline where the total allocated 
budget is in excess of the contract value (contract budget base) or project baseline. It 
may involve replanning in-process work, and/ or adjusting variances. 

Performance Measurement Baseline (PMB) . The time-phased budget plan against 
which performance is measured. It is formed by the budgets assigned to scheduled 
control account and the applicable indirect budgets. For future effort, not planned to 
the control account level, the performance measurement baseline also includes budgets 
assigned to higher level WBS elements and undistributed budgets. It equals the total 
allocated budget less management reserve. 

Planned Value . Same as Budgeted Cost for Work Scheduled. 

Planning Package (PP) . A logical aggregation of work within a control account, 
normally the far-term effort, that can be identified and budgeted in early baseline 
planning, but is not yet defined into work packages. 
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Post Acceptance Review . A government review performed on a specific element or 
elements of a performing activity's EVMS system that display (s) a lack of discipline or 
no longer meet(s) the intent of the EVMS Guidelines. 

Price Variance (PV) . The portion of a material cost variance due to price change. The 
difference between the planned unit cost of materials and the actual unit cost of 
material. PV is derived by subtracting the planned unit price times the quantity used 
from the actual unit price times the quantity used. 

Program (or Project or Product) Manager (PM) . The person assigned the prime 
responsibility for overall management of a program (or project, or product). 

Progress Payments . Payments made to a contractor or subcontractor during the life of 
a fixed price type (firm fixed price and fixed price incentive) contract on the basis of a 
percentage of total incurred cost. 

Quantity Variance (QV) . (See Usage Variance). 

Recurring Costs . Expenditures against specific tasks that occur on a repetitive basis. 
Examples are costs of sustaining engineering support, repeated fabrication or assembly 
of parts or products, tool maintenance, etc. 

Replanning . The process by which a program or project updates or modifies its plans. 
This applies to a change in the original authorized PBB or CBB planning for 
accomplishing formally authorized requirements, typically involving the redistribution 
of budget for remaining work. In accordance with the ANSI/EIA-748, traceability is 
required to previous baselines, and funding requirements need to be considered in any 
replanning effort. There are two types of replanning: 

• Internal Replanning. Replanning actions performed by the supplier for remaining 
effort within the recognized Project Budget Base or CBB. It is caused by a supplier's 
need to accommodate cost, schedule, or technical problems that may have made the 
original plan unrealistic. Internal replanning is restricted to remaining effort and if 
significant, the customer must be advised of the action. 

• Authorized Change (or External) Replanning. A change necessitated by 
government/ customer direction which may be in the form of either a definitized or 
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a no cost contract change order for contracts or formal change to the Project Plan for 
in-house Projects that calls for a change in the original plan. It most often results 
from a change in the authorized requirement affecting cost, schedule, technical 
parameter or a combination thereof. 

Reprogramming (or Formal Reprogramming). A comprehensive replanning of the 
remaining PMB that result in an Over-Target Baseline (OTB), an Over-Target Schedule 
(OTS) or both. This type of replan is for performance measurement purposes only and 
requires prior coordination and approval of the Customer. 

Risk Analysis . The system that provides a continuous analysis of identified risks with 
respect to their impact on project cost, schedule, and technical performance. 

Rolling Wave Planning . The progressive refinement of work definition as time goes on 
by continuous subdivision of downstream activities into detailed tasks. 

Rubber Baselining . Actions by the supplier to advance far-term budgets into the 
current or early periods to mask current cost problems. The action involves moving 
budget without a corresponding amount of task, to cover current cost difficulties. It is 
an indication of likely overrun condition. 

Schedule . A plan that defines when specified work must be started, worked on, and 
finished, to accomplish program objectives on time. 

Schedule Performance Index (SPI) . A measure of schedule efficiency. It compares the 
BCWP to the work scheduled (SPI = BCWP / BCWS). An index of 1.0 means the work is 
being performed right to the schedule. SPI > 1.0 means that the work is ahead of 
schedule. SPI <1.0 means that the work is behind schedule. 

Schedule Variance (SV) . A metric for the schedule performance on a program. The 
algebraic difference between the earned value (BCWP) and the budget plan (BCWS). 
(SV=BCWP-BCWS) A positive value is a favorable condition while a negative value is 
unfavorable. It may be expressed for a specific period or cumulative to date. 

Statement Of Work (SOW) . The document that defines the work scope requirements 
on a program or contract. 
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Subcontract . A contract for services, data, parts, components, assemblies* other 
hardware* or software that a company commits to perform for or provide to the prime 
contractor. A subcontract normally involves the design or production of a component 
by the supplier to the prime contractor's specifications. 

Summary Level Planning Package (SLPP) . An aggregation of work for far-term 
efforts, not able to be identified at the control account level, but which can be assigned 
to higher level WBS elements (and is therefore not "undistributed budget"). 
Surveillance . The continuous process of reviewing the health of the earned value 
management system (EVMS) as applied to one or more projects. 

Surveillance Plan . An annual plan that identifies the projects to be included in 
surveillance reviews, as well as the frequency and scope of the individual surveillance 
visits planned for each project included in the annual plan. 

Task . A piece or portion of discrete, apportioned, or level-of-effort work. Also called 
an activity, something that takes place over a period of time and generally consumes 
resources. 

Thresholds . Boundaries or limits (monetary, time, or other values), which, if breached, 
result in some type of management review and action. 

To-Complete Performance Index (TCPI) . The future cost efficiency needed to 
accomplish the remaining work within a financial goal such as the Budget at 
Completion (BAC) or the Estimate at Completion (EAC). It compares the budget for 
remaining work with the remaining cost or the estimated remaining cost to complete 
the work. TCPIBAC = (BAC - BCWP cum) / (BAC - ACWP cum). Or TCPIEAC = (BAC 
- BCWP cum) / (EAC - ACWP cum). Compare the CPI to determine if the BAC or the 
EAC is realistic or not. 

Total Allocated Budget (TAB) . The sum of all budgets allocated to a project/ contract. 
Total allocated budget consists of the PMB and all MR. The TAB should reconcile 
directly to the project/ contract budget base. If the TAB is greater than the 
project/ contract budget base, the difference is attributable to an over target baseline 
and must be documented. 
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Undistributed Budget (UB) . Budget associated with specific work scope or authorized 
changes that have not been assigned to a control account or lower level WBS element. 
Unpriced Changes . Authorized changes that are not yet priced. 

Usage Variance (UV) . The portion of material cost variance due to a change in quantity 
of material used. The difference between planned quantity of materials and the actual 
quantity used, expressed in dollars. UV is derived by subtracting from planned 
quantity times planned unit cost, the actual quantity times planned unit cost. (Same as 
Quantity Variance.) 

Variance Analysis Report (VAR) . An internal document, within an earned value 
management system, for the analysis and reporting of variances which breech the 
established thresholds. It requires the reason or cause of the variance; the impact on 
cost or schedule; and the corrective action required, or accomplished, on significant 
variances. Format 5 of the CPR is a VAR. 

Variance At Completion (VAC) . The difference between the total budget assigned to a 
contract, project, WBS element, organizational entity, control account, or work package 
and the estimate at completion. It represents the amount of expected overrun or 
underrun. (VAC equals BAC minus EAC, i.e., total allocated budget minus the related 
estimated final cost). 

Variance Threshold . Internal and external tolerances (or thresholds), which are 
established by management direction, or negotiation with the customer and which, 
when exceeded, require investigation, analysis, reporting and corrective action. 

Work Breakdown Structure Element . A single discrete portion of a WBS. May be an 
identifiable product, a set of data, or a service. 

Work Package (WP) . ). A detail, short duration task or material item identified by the 
Project Control Account manager for accomplishing a Control Account task. A work 
package has the following characteristics: 

• Represents unit of work at the level where work is performed. 

• Clearly separate from other Work Packages. 
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• Assignable to a single organizational element. 

• Has scheduled start and completion dates, and interim milestones, if required, all of 
which represent physical accomplishment. 

• Has budget expressed in terms of dollars or hours/ FTEs. 

• Its duration is limited to a relatively short span. 

• Is integrated with detailed engineering, shop, or other schedules. 

• Has a correct Earned Value Technique assigned to it. 

Work Package Budgets . Resources that are formally assigned by the contractor to 
accomplish a work package, expressed in dollars, hours, standards or other definitive 
units 
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APPENDIX G IBR TRAINING 




EARNED VALUE MANAGEMENT / 
AND INTEGRATED BASELINE 
REVIEWS 


IBR Training Agenda 



■ General IBR Process 

■ Earned Value Management Overview 

■ IBR Overview 


2 
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INTEGRATED BASELINE REVIEW 
PROCESS 


3 


Importance of an IBR 



IBR lays a solid foundation for project execution and increases the 
odds of successful project completion 


□ Opportunity to compare project manager(s) expectations and to address 
differences before problems arise 

□ Understand the project plan and risks - target resources to meet project 
challenges 

□ Increased confidence in the project cost, schedule, and performance data - 
a tool for proactive project management and the capability to make timely 
and reliable cost and schedule projections 
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Intent of IBR 


■ The IBR should confirm that the Performance Measurement 
Baseline (PMB) 

• captures the entire technical scope of work 

• work is scheduled to meet the project objectives 

• risks are identified 

• the proper amount and mix of resources have been assigned to 
accomplish all requirements 

• management control processes are implemented. 

Recognize that the IBR is an event, however the purpose and 
objectives should be viewed as a continuing process to 
ensure mutual understanding of the PMB and associated 
risk. 
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Assessment Process 


A PMB that is complete and documented at the appropriate 
level of detail is a key element for project management to 
successfully achieve project objectives. 


Focus on risks that may impact project planning and execution 




IBR Team Structure 
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T raininq/Preparation 


■ The intent of the training is to provide enough information so the 
team can understand the cost, schedule, technical, and 
management processes used on this project. 

. Scope of Work/Statement of Objectives 
. Budget Baseline 

• Schedule Baseline 
. Funding 

. EVM Basics 
. IBR Conduct 

• Project Managers’ Expectations 
. Project Management Processes 

. Risk Identification and Documentation 
. Subcontractor(s) 



IBR Execution 


The focus when conducting the IBR is the mutual 
understanding of the PMB content and the risk. Anything 
that does not support the purpose should be moved outside 
the review. 


The key event is the control account managers discussions 


- The discussions should focus on the key aspects of the 
PMB. 


To be effective the discussion team must remain small, 
focused, and composed of knowledgeable participants 


10 
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Identify Risks 


■ Review and incorporate into the project’s risk management 
plan: the technical, cost, and schedule risk as identified in 
the PMB, and the risk of the management processes to 
successfully execute the PMB. 

■ To identify project risks the following items must be 
assessed: 

• Scope: impact on product meeting performance requirements and funding 
authorization 

• Schedule: Logical flow concerns, Task dependency gaps, Adequate time to 
accomplish 

• Resources: Supports schedule & scope to meet performance requirements 

• Management control processes: the degree in which effective 
cost/schedule/technical planning and control processes exist and are 
implemented, 

• Management Reserve (MR): should also be compared against risk 
assumptions. The assessment should determine whether there is sufficient 
management reserve to address the risk items identified. 

11 



IBR Closeout 


■ After completion of the IBR discussions there needs to be a 
closure plan. 

• The Project Manager should agree on a plan of action and who is 
responsible for the action. 

«■ Risk items identified need to be classified as to the severity of the 
risk involved. 

Those items identified, as Action Items require Project Managers’ 
attention and should be documented in the Risk Management 
Plan. 

3- Items identified as Watch Items represents concerns that may 
require future attention. 
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Summary 



■ The IBR is an event, however the purpose and objectives should be viewed as a 
continuing process to ensure mutual understanding of the PMB and associated 
risk 


■ The purpose of an IBR is to achieve a mutual understanding of the risks inherent on 
the PMB and the management control processes that will operate during execution 

■ IBR preparation and planning are the key to a successful review 

■ A PMB that is complete and documented at the appropriate level of detail is a key 
element for project management to successfully achieve project objectives 


The focus when conducting the IBR is the mutual understanding of the 

and risk. Anything that does not support the purpose should be m 
review 


Evaluate the integrated PMB to determine whether it, captures the ei 
is consistent with authorizing documents, and has adequate resources 1 
requirement and planned project tasks 




BASIC EARNED VALUE 
MANAGEMENT 


14 
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EVM & Integrated Product Teams 
The Key to Success 



0 Management systems don’t manage - people do! 

0 EVM is used to identify, communicate and manage the 
resource effect of technical and schedule problems. 


COST 

8 



TECHNICAL 
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Traditional Cost Management 



Spend Plan vs. Actual 


OR Budget vs. Actuals 




900 

800 

700 

600 

500 



Project Budget 
Actual Cost to Date 


50 % Spent 

But What Does that Really Mean? 


100 

0 



JFMAMJJASOND 


16 
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Actual Cost 




Well, I’ve spent 400 hours, 
does that mean I’ve 
accomplished 400 hours 
of work? j 


Actual Cost is not an indication of work progress, 
but only an indicator of hours/money spent 


17 


Contract Performance 




18 
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Three Steps to Success... 




• identify what is to be done in terms of tasks and underlying 

assumptions 


specify when the work is to begin and end in terms of order 
and duration 


define who will do the work by identifying resources (people, 
equipment, and costs) and constraints 


19 


Defining the Work Scope 




identify what is to be done in 
terms of tasks and underlying 

assumptions 

• Begins with the SOW/SOO 

• Project Team expands 
SOW/SOO tasks into work 
authorization documents 

• Control Account Managers 
(CAMs) “negotiate” the work 
scope 

• Everything must be covered 



20 
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Schedule the Work 



specify when the work is to begin and end in terms of 
order and duration 

• Order = sequence of work 

Finish to start (start to start, finish to finish) 

*■ Leads and Lags 
Added details may reduce risk 

• Duration 

Total time to complete a task 
»■ Resource impacts must be considered 
Risk in schedule? 


Schedule Integration 



PROJECT LEVEL 


The master schedule and 
lower level schedules must 
be traceable both 
VERTICAL and 
HORIZONTAL 


These two levels of 
schedules do not 
show horizontal 
relationships. 


DETAIL PROJECT LEVEL 



PROJECT 

SCHEDULE 


ITEM 

y 

2 

3 

4 

5 

x> 

7 

COST ACCOUNT 14 
COST ACCOUNT 15 
COST ACCOUNT 16 
COST ACCOUNT 17 
COST ACCOUNT 18 
COST ACCOUNT 19 

s 


. s 
s 



s 

\ 

s 



s 

r 



N S 


a- 

s. 


MAJOR EVENT 
OR FUNCTIONAL 
ORGANIZATION 
MILESTONE 
SCHEDULE 



COST ACCOUNT 
AND WORK 
PACKAGE 
SCHEDULES 
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Identify Resources 



Period Period Period Period 
12 3 4 


Engineer 

50 

22 

130 

75 

Senior Engineer 

15 

67 

50 

11 

Principal Engineer 

20 

38 

51 

28 

Systems Engineer 

5 

64 

20 

19 


Resource requirements shown in hours, direct cost, or total cost. 
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The Control (Cost) Account 



■ A management control point consisting of work 
packages and planning packages, where 
accountability for performance is measured 

■ Other characteristics of a control account are: 

• BCWS, ACWP and BCWP are integrated! 

• Variances Are Assessed 

• Estimates Are Revised 
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EVM Basics 

Control Account Elements 



Work Packages 

Detailed, short-span tasks, or 
material items, required to 
accomplish the CA objectives, 
typically in the near term. 


Planning Packages 
Future work that has not been 
detail planned as work 
packages. They are always 
scheduled to occur in the future. 



* During the IBR, we look at how earned value was set up, for each task, 
and how it will be taken. 
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Classifvina the Work 


■ Discrete or Measurable 


• Specific end product or result 

• Preferred category of work because objective 


• EV methods include 0/100, milestone, percent complete, etc. 

■ Level of Effort (LOE) 


• Has no specific product 

• Measured by the passage of time 


■ Aooortioned effort 


• Dependent on other work, discrete tasks 

• Measured as a factor, e.g., 10% of discrete task 

26 
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EVM Basics 

Earned Value Terminology 


Data Element 

Term 

Acronym 

Scheduled Work 

Budgeted Cost for Work Scheduled 

BCWS 

Earned Value 

Budgeted Cost for Work Performed 

BCWP 

Actuals 

Actual Cost of Work Performed 

ACWP 

Authorized Work 

Budget At Completion 

BAC 

Forecasted Cost 

Estimate At Completion 

EAC 

Work Variance 

Schedule Variance (BCWP-BCWS) 

SV 

Cost Variance 

Cost Variance (BCWP-ACWP) 

CV 

Completion 

Variance At Completion (BAC-EAC) 

VAC 


The Language of Earned Value Management 


■ Control Account - the key management control point at which 
ACWP, BCWP, and BCWS are managed 

■ Work Packages - detailed planning of short span jobs or material 
items representing units of work at levels where work is 
performed and BCWP is measured 

■ Planning Packages - a logical aggregation of work which has not 
been detailed planned into work packages 

■ Rolling Wave - period of time within which detailed planning must 
occur 

■ Measurable or Discreet Effort - contract effort that produces 
definable end products 

■ Level of Effort (LOE) - contract effort that does not produce 
definable end products 

■ Apportioned Work - effort that be itself is not divisible into work 
packages 
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The Language of Earned Value 
Management (cont.) 

■ Estimate At Complete (EAC) - ACWP, plus an estimate of costs 
for remaining work 

■ Budgeted Cost for Work Scheduled (BCWS) - budget for 
scheduled effort 

■ Budgeted Cost for Work Performed (BCWP) - the value of the 
work performed 

■ Actual Cost of Work Performed (ACWP) -incurred and recorded 
costs within a given time period 

■ Cost Variance (CV) - the difference between BCWP and 
ACWP (CV = BCWP- ACWP) 

■ Cost Performance Index (CPI) - the average cost efficiency with 
which work has been performed. (CPI = BCWP/ACWP) 

■ Schedule Variance (SV) - the difference between BCWP and 
BCWS (SV = BCWP - BCWS) 

■ Schedule Performance Index (SPI) - the ratio of the value of work 
performed to the value of work scheduled (SPI = BCWP/BCWS) 29 



Creating the Performance 
Measurement Baseline (PMB) 




30 
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Completing the Picture 




2 

Days 


4 6 


8 10 12 14 16 18 20 ^ 

Report Date-^ Completion dat^ 



INTEGRATED BASELINE REVIEWS 
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Preparing Yourself 


■ Know your CWBS assignments 

• Be familiar with scope, schedule, budget 

■ Have an objective. What do you expect to gain from the 
discussion? 

■ What questions will you ask to achieve the objective? 

■ Prepare a tentative list of questions to serve as a framework for 
the discussion. 

■ Become familiar with the documentation. 

■ The multi-functional Team (Cost, Schedule, Technical) is an 
opportunity to transfer knowledge and experience 
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What Documents Will Be Reviewed? 



■Work authorization documents 


■ Control (cost) account plans 


1 ^ ^ 

■ Schedules 

9 



■ Cost reports 

■ Project Work Breakdown Structure (PWBS) 
and Dictionary 
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During the Discussion 



Keep In Mind... 

■ Introduce yourself and identify the organization you represent. 

■ Ask the CAM to walk through his/her notebook. 

■ Be friendly and non-confrontational. If there appears to be 
confusion, restate your question. Clarify any misunderstandings. 

■ Take notes during the discussion. Notes will be used to 
complete the Discussion Assessment Form and Concern Area 
Reports (CARs) after the interview. 

■ If necessary documentation is not readily available, get a 
commitment of where and when the documents will be available. 
Fill out Documentation Request Forms when necessary. 

■ Watch the time, but do not leave until you are satisfied the 
discussion is complete. 

■ If disagreements arise, do not argue with the Project Team. 

Write a CAR to the Team Leader, and let the Team Leader 
handle any continuing discussion with the Project Team. 
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IBR - CAM* Discussions 



Technical Assessment: 

• What are the high risk areas 

• Conditions of measurement 

• What are the technical requirements 


Schedule Assessment: 

• Underdeveloped logic 

• Illogical/Forced Sequencing 

• Lack of Experience 

• Consideration of Technical Risks 



Budget Assessment: 


Work Authorization: 

• Schedules 

• Budget Authorization 

• Control Account Plans 


• Budget Validity/Risk 

• Distribution Logic 


• Management Reserve 

• Undistributed Budget 



Work Assessment: 

• Which Tasks may be Overstated? 

• Which Tasks may be Understated? 

• What Tasks have been left out 


* Control Account Manager (CAM) 
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The Discussion -- Work Scope 



■What is your scope of work? 

■ How did you plan your work into control (cost) 
accounts? 

■ How were resources obtained? 

■ Please describe the resource development 
process. 

■What is your experience on EVMS 
projects? 

■ Is all of your work authorized? 


37 


The Discussion -- Schedule 



■ Can you discuss 

• Identification and management of schedule impacts? 

• Who owes you things and how do you track them 

• How your work relates to project milestones 

■ Please describe known risks in your plan. 

■What assumptions are built into your 
schedule? 


38 
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Common Schedule Presentations 



■ Commonly used “schedule presentations” have strengths... and 
weaknesses 

• Gantt Charts 

• Milestone Charts 

• Network Schedules 




Resourcing the Schedule- A Reality Check 


■Categories of labor - do they support the 
task? 

■ Labor availability - are labor resources 
available during the period of performance? 

■Over/Under Allocation of Resources 
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The Discussion -- Budget 



■ How was the budget for your work developed? 

■ Is your budget adequate? 

■ If your budget is inadequate, what are your 
options? 

■ Please explain the time-phasing of your 
budget. 

■ How do you develop estimates to co 

■ Please explain the particular earned 
method(s) and why they were sele 



Measuring Performance 



■Work Progress quantified through an objective 
measure 

■ Performance must be computed using the 
same labor, overhead, and other rates as 
scheduled 

■ Provides a measure of work accomplishment 
against the plan 
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Identify Labor/Overhead Rate Impacts 



■ Refer to System Description and Accounting 
Disclosure Statement 

■ Discuss with DCMA/DCCA recent 
labor/overhead Rate Trends 

■ Cost Accounting Standards Compliance 
issues? 

■What are the pools? 

■ How are indirect monitore' 

<*- Actuals 
«■ EAC 



After the Discussion 


■ Complete the Discussion Assessment Form after each interview. 

■ Complete Concern Area Report(s), Documentation Request 
Forms, and General Assessment Form(s) if applicable. 

■ Submit all forms to the Sub-team leader. 

■ Be sure to follow up. Do not leave any loose ends hanging. 
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Sample Findings 


■Technical 

• Scope unaccountable 

• Non-integration of cost and technical performance 

• Subcontractor requirements flow down 

• Lack of work around plans for high risk areas 

• Project Team plans not supporting critical milestones 

• Tooling not being produced quickly enough 

• Software concerns (type of language, productivity factors, 
and amount of software being written) 

• Specification issues 
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Sample Findings (cont.) 


■ Project Management 

• Schedules not linking 

• Delayed subcontract awards 

• Communication problems 

• Inadequate time scheduled 

■ Cost 

• Inadequate budgets 

• Time phasing of budgets 

• Unreasonable productivity factors 

• Lack of rework budgets 

• Manager knowledge 

• Unreported future cost impacts 

• Performance measurement issues 

• Management reserve « 
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Lessons Learned -Before Review 



■ Subteams should be no larger than 5-6 
members 

■ Knowledge of critical path essential in 
identifying risk areas 

■A large portion of the work is done prior to 
going 
on-site 

■A well trained team is essential 

■ Selection of sub-team leader is very 
important 

■ Risk areas should be identified early to allow 47 


Lessons Learned - During Review 



■Coordinate the IBR with other project reviews, 
when practicable 

■Stay focused on the current contract 

■ Document manager discussions immediately 
after interviews/discussions 

■ Do not turn the review into a design review 
(review is to identify resource/planning 
problems not solve them) 

■Conduct an on-site IBR Team meeting 

■Conduct a maximum of 4 discussions per day 
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APPENDIX H 


IBR IN-BRIEF TEMPLATE 


<Project Logo> 


<Contract # or Subproject Name> IBR In-brief 


<Project Name> Project IBR Team 

<Date> 

<Location> 



In-brief Agenda 


IBR Purpose 
IBR Objectives 

£ 

<L) 

IBR Approach & Methodologies 

CU 

CH 

IBR Team Members 

<L) 

C 

q3 

Discussion Agenda 

ru 

CO 

-Q 

IBR Scope 

cu 

-t-* 

03 

5 — 

Discussion Schedule 

CU 

+-» 

C 

Questions & Answers 
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Purpose of the Review 

We ARE conducting the IBR to evaluate the adequacy of the PMB 
and identifying concerns. 

We ARE NOT looking for problems to solve. 

We jointly need to be able to answer this basic question... 

Can we execute the technical work scope of this contract given 
the available schedule, budget and other relevant resources? 


In other words, do we have a plan and is it feasible? 


The Objectives of This IBR 


QJ 

■> 

cu 

cc 

cu 


ao 

cu 


Validate and understand the PMB 

x Is the entire technical scope of work represented? 
x Can the SOW be accomplished? 
x work is scheduled to meet the program objectives? 

x Are the proper amount and mix of resources have been assigned to 
accomplish all requirements? 

Identify risks to the PMB 
x Technical 
x Cost 
x Schedule 
x Resources 

x Management processes 
Perform corrective actions now 
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IBR Approach and Methods 

Control Account Managers discussions 
Technical or IPT Managers summary discussions 
IBR Team discussions & evaluations 
System and Baseline Documentation Reviews 
Data traces (sampling only) 

Process reviews - 

x Scheduling and Risk/EAC 
x Change Control 
x etc. 

It is important to recognize that the IBR is an event, however the 
purpose and objectives should be viewed as a continuing process to 
ensure mutual understanding of the PMB and associated risk. 


Method for Risk Analysis 


What is the likelihood the risk will happen? 

Level 

Planned Approach and Processes 

1 

Not Likely 

Will effectively avoid or mitigate this risk 
based on standard practices 

2 

Low Likelihood 

Have usually mitigated this type of risk with 
minimal oversight in similar cases 

3 

Likely 

May mitigate this risk, but work arounds will 
be required 

4 

Highly Likely 

Cannot mitigate this risk, but a different 
approach might 

5 

Near Certainty 

Cannot mitigate this type of risk, no known 
processes or workarounds are available 



Given the risk is realized, what would be the magnitude of the impact? 


Level 

Technical 

Schedule 

Cost 

1 

Minimal or no impact 

Minimal or no impact 

Minimal or no impact 

2 

Minimal performance 
shortfall, same approach 
retained 

Additional activities 
required, able to meet 
key dates 

Budget increase or unit 
production cost 
increase <1% 

3 

Moderate performance 
shortfall, but work arounds 
available 

Minor schedule slip, will 
miss key dates 

Budget increase or unit 
production cost 
increase <5% 

4 

Unacceptable, but 
workarounds available 

Program critical path 
affected 

Budget increase or unit 
production cost 
increase <10% 

5 

Unacceptable, no alternatives 
exist 

Cannot achieve key 
program milestone 

Budget increase or unit 
production cost 
increase >10% 


Consequence 


Pre-Mitigation Risk 
Risk w/mitigation 


Current Status 


Integrated Baseline Reviews 
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Risk Type Definition 


Technical: ability of the project's technical plan to achieve the 
objectives of the scope of work 

Schedule: adequacy of the time allocated for performing the 
defined tasks to successfully achieve the project schedule 
objectives 

Cost: ability of the PMB to successfully execute the project and 
attain cost objectives, recognizing the relationship between 
budget, resources, funding, schedule, and scope of work 

Resource: availability of personnel, facilities, and equipment, 
when required 

Management Processes: degree to which the management 
processes provide effective and integrated 
technical/schedule/cost planning and baseline change control 


IBR Team Members - <Project Name> 


QJ 

> 

QJ 

cc 


~o 

CD 


CtO 

QJ 


Role 

Team Member 

Project Manager 



IBR Team Leader (Deputy PM) 
IBR Coordinator 
IBR Facilitator 

EVM/Schedule Management 
Cost/Budget Analyst 

Sub-team Lead <Technical Area> 
Sub-team Lead <Technical Area> 
Sub-team Lead <Technical Area> 
Sub-team Lead <Technical Area> 
Sub-team Lead <Technical Area> 
COTR 
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Agenda 


Tuesday, Month Day, Year > 


8:00-9:00 

IBR In-briefing, Administrative Details 


9:00-10:15 

New Documentation Review 


10:30-12:00 

Discussion Sessions 1 

CD 

12:30-1:30 

Lunch 

CD 

1:30-300 

Discussion Sessions 2 

CC 

CD 

3:00-5:00 

Government Team Review 

C 

<Wednesday, Month Day, Year > 

LO 

CD 

8:00-9:30 

Discussion Sessions 3 

CD 

-Q 

10:00-11:30 

Discussions Sessions 4 

CD 

+-> 

12:00-1:00 

Lunch 

CD 

i_ 

tUO 

1:00-2:30 

Discussion Sessions 5 

CD 

+-> 

c 

2:45-5:00 

Government Team Review 


<Thursday, Month Day, Year > 


8:00-12:00 

Government Team Review - Exit Outbrief Preparation 


12:00-1:00 

Lunch 


1:00-1:30 

Supplier Manager Debrief 


1:30-2:30 

IBR Exit Outbrief 


Scope of IBR 


QJ 

> 

CD 

CH 


of> Discussions and Baseline documentation from 
Sampling from <#of> Control Accounts: 

x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
x <CAM - Control Account # - Control Account Title> 
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Discussion Schedule 



CAM 

Sub-Team Lead 

Location 






c 

o 

Tuesday - 10:30a 




(o 

0) 




lo 





C\| 





c 

o 

<Date & Time> 




Cb 









rq 





C 

o 

<Date & Time> 




Cb 









SJ- 





c 

.0 

<Date & Time> 




<0 

0) 









LT) 





c 

0 

<Date & Time> 




£ 
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Discussion Schedule 



CAM 

Sub-Team Lead 

Location 

Session 1 \ 

Tuesday - 10:30a 













Session 2 \ 

<Date & Time> 













Session 3 \ 

<Date & Time> 













Session 4 

<Date & Time> 













| Session 5 

<Date & Time> 
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APPENDIX I 


DOCUMENTATION AND DATA TRACE EVALUATION 
TEMPLATE 


Documentation & Data Trace Evaluation 


Date: 


Documentation Inventory: 


Organization 

Does the WBS and related documentation detail the Control Account Work scope? 

Do the Org charts correctly reflect the CAM Organization? 

Control Account Plans 

Is the CAM and relevant WBS/OBS elements adequately referenced? 

Are there any discrepancies between the WBS/OBS and CA/CAP references? 

Does the CAM Name match that shown on the OBS Chart? 

Do the hour and dollar values at the Control Account Level on the RAM/CA 
Statement of Work and Budget Allocation documents match those in the CAP? 

Do the breakdowns for ODC, materials and sub contractor costs etc match those approved at 
the summary level in the CAP? 

Do all the Open CAs/WPs have Cost Collection Numbers? 

Do any of the PPs have CCNs? 

Do any PPs have an 'Open' Status? 

Do all the WP and PP's have SoWs? 

Do all the WP and PPs have Budgets? 

Are the WBS roll up's accurate? 

Are there duplicate WBS numbers? 

Are WPs/PPs defined at an appropriate level in each project area? (Check for duplicate 
packages (titles / sows))? 

Do the start and Finish dates on the schedule match those in the CAP? 

Do the schedule/project titles/activity names etc, correspond to the WBS? 

Was the planning process appropriate and in line with planning guidelines (project 
instructions)? 

RAM 

Does the RAM detail the same WBS as the WBS and OBS charts? 

Do the Dollars, Items and Hours match the CAP front sheet? 

Schedule 

Do the Control Account Schedules tie to the IMS? 

Are all tasks within the CA and Contract Period of Performance? 


Yes 

No 








































Yes 

No 
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Additional Comments 



The Documentation and Data Review has been carried out by: 


Name: Role: 


Name: Role: 


Name: Role: 


Name: Role: 


Name: Role: 


116 




APPENDIX J 


IBR LOGS 


Action Item Log 


CAR# 

CAM / Manager 

WBS/CA 

Brief Description 

Action Description 

Responsible Team 
Member 

Target Close 








































































CAM Discussion Log 


CAM/ 

Manager 

Responsible Area 

Discussion Eval # 

Risk Eval # 

Document Request #'s 

CAR #'s 

Sub-team 

Lead 
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APPENDIX K OUTBRIEF SAMPLES 


Integrated Baseline Review 
Sample Out-Brief 


Project Name 


IBR Objectives 


> Validate and understand the PMB 

> Can the SOW be accomplished? 

> Are adequate resources, schedules and budgets assigned? 

> Identify risks 

> Technical 

> Cost 

> Schedule 

> Resources 

> Management processes 

> Develop mitigation plans 

> Give Project Managers early options 
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Project Risk Summary 


IPT 

Technical 

Schedule 

Cost 

Resources 

Mgmt 

Processes 

Aeronautical & 
Spacecraft Systems 

M 

M 

L 

M 

M 

Systems l&T 

L 

L 

L 

L 

M 

Payloads 

L 

L 

M 

L 

M 

Mission Ops 

M 

M 

M 

M 

M 

Launch 

Vehicle/Services 

M 

L 

L 

M 

M 


Low Risk: Little potential to disrupt schedule, increase costs, or degrade performance... no significant 
issues, the plan is executable. Management processes are adequate, understood and effectively used. 
Normal government monitoring may overcome difficulties. 


Medium Risk: Can potentially disrupt schedules, increase costs, degrade performance. . .some 

significant issues but the plan remains executable. Management processes are generally adequate, 

understood and effectively used. Special contractor emphasis and close government monitoring can 
overcome difficulties. 

High Risk: Likely to significantly disrupt schedules, increase costs, degrade performance. . .significant 

| | issues impacting plan execution. Management processes are not adequate, understood or effectively 

used. Risk is unacceptable even with emphasis and close government monitoring. 


Aeronautical & Spacecraft 
Systems 



> Technical: Historical instability issues specific to the SW infrastructure may be 
exacerbated in Build 2.4 due to functional complexity. (Reference Risk 02-026) 

> Schedule : Risk mitigation steps established to reduce Build 2.4 schedule risk have 
not been fully realized due to late delivery and inability to take early looks at weapons 
functions. (Reference Risk 02-026) 

> Cost: Cost performance is within budget and adequate budget is available to manage 
contingencies. 

> Resources : Lab assets are strained. 

> Management Processes : Baseline planning and earned value method used are not 
consistent with expected workload/complexity. The CPR is not providing enough 
insight into developing cost/schedule problems. This impacts management’s ability to 
effectively use data. Have seen improvements in other processes such as estimating 
and metrics. 


4 
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Systems Integration & Test 



> Technical: No significant concerns noted. 

> Schedule : No significant concerns noted. 

> Cost: $1 .8M of additional personnel may be off set by reduction of test 
points. 

> Resources : All resources to execute the test program have been identified 
and planned. 

> Management Processes : Management processes with the exception of 
change control are robust and provide early indications of problems. 
Approximately $1 .8M of MR was issued to work already in progress. 
Management team seems to have a good grasp of requirements. 


5 


Payloads 



> Technical: No new unforeseen performance risks identified. Existing watch items of 
note: 02-023, 00-201 A. 

> Schedule : No new schedule drivers identified. Moderate risk reflects current state of 
risks already in the database. 

> Cost: No new cost drivers identified. Moderate risk reflects current state of risks 
already in the database. 

> Resources : No new issues identified. Potential still exists for test asset shortfalls. 

> Management Processes : EVM credit methodology is not sufficiently objective. 
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Mission Operations 


Technical 

Schedule 

Cost 

Resources 

Mgmt Process 







> Technical: No significant concerns noted 

> Schedule : No significant concerns noted 

> Cost: There are potential additional costs to program if invoices are larger than 
current budget 

> Resources : No significant concerns noted 

> Management Processes : Noted problems with material time phasing and EVM 
methodology. Invoice problems with subcontractor need to be resolved. 


Launch Vehicle/Services 



> Technical: Moderate risk (1 6) reflects current state of risks already in the database. 

> Schedule : No significant concerns noted. 

> Cost: Adequately budgeted. 

> Resources : Overtime required to meet accelerated schedule. 

> Management Processes : No objective or documented quantifiable backup data for 
EVM methodology. Horizontal integration needs to occur at the work package level. 
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Safety & Mission Assurance 


Technical 

Schedule 

Cost 

Resources 

Mgmt Process 







y Technical: There is no documented traceability from SOW to product 
and task descriptions. 

> Schedule : Government team planning to review Basis of Estimate to 
ensure realistic planned durations. 

> Cost: No concerns noted. 

^ Resources : Current resources are not accomplishing tasks in a timely 
manner. 

> Management Processes : evm methodology is a good model tor 
objective determination and supporting documentation. 


Total Program 



> Technical: There is a good understanding of technical risks and mitigation plans are in 
place. 

> Management Processes : The review noted several systemic issues related to EV 
Management: 

- EVM methodology 

- Reprogramming adjustments 

- CAM responsibility - LRE process, material budget 

- Variance Analysis process 
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New Risks 


> Aeronautical & Spacecraft Systems 

■ Insufficient time to correct late-breaking priority fixes during test. 

■ Current system architecture limits future growth capability as well as existing R&M, 

■ Need to provide requirements definitions to subcontractor. 

■There is no budget identified for S/W support during test to include correction of 
showstoppers. 

■ The Lab schedule does not exist in enough detail to ensure maximum use of 
resources. 

> Systems Integration & Test 

■ Testing procedures at lower levels have not yet been defined. 


11 


New Risks 


> Payload 

■Current weight estimates above allowable weight. 

■Payload may require special support equipment for placement on the vehicle. 

>Mission Operations 

■ Additional Ground Support Equipment may be required. 

■ Unanticipated modifications to existing facilities to support processing. 
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Documents Needed 


> Basis of estimate for Supportability 

> Time Phased BCWS in $’s by element of cost 

> Risk Management Plan 

> Control Account Plans 
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General Observations 


> Total Concern Area Reports (CARs) generated: 34 

> New Risks Identified: 10 

> Management Process Systemic Issues 

- Noted improvements but EVM issues still need to be resolved 

- Schedules not resource-loaded 

> Commitment from both Teams: 

- Strong Government/Contractor team relationships 

- Concerted effort to understand PMB 

- Tools, processes in place to help managers 

• PMM, 

• Cost/Schedule Comparison; new system on the way 

• Schedule metrics 


14 
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Outstanding Actions 


> NASA IBR Team : 

- Assess risks; update existing risks and/or add new risks to Risk 
Management Plan (Due 28 OCT 05) 

- Formal IBR Letter to contractor/in-house organization with copies of 
Concern Area Reports (Due 31 OCT 05) 

> Contractor/ln-House Organization : 

- Provide requested documentation (1 8 OCT 05) 

- Written response to Concern Area Reports (30 NOV 05) 

> IBR Team Leader & Review Facilitator : 

- Evaluate responses to CARs and conduct follow-ups if required. (15 DEC 
05) 


15 


Integrated Baseline Review 
Sample Out-Brief 


<Project Name> 
Integrated Baseline Review 
<IBR Date> 
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IBR Goals 


> Achieve a mutual understanding of the baseline 
plan and its relationship to the underlying Earned 
Value Management System (EVMS) 

> Assess the adequacy of the performance 
measurement baseline (scope, schedule, budget, 
resources, and management processes) 


SCOPE of IBR 


> Evaluate the current baseline and 
the associated risks 
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AREAS ASSESSED 


> Program Management 

> Aeronautical & Spacecraft Systems 

> Safety & Mission Assurance 

> Payloads 

> System Integration & Test 

> Launch Vehicle/Services 

> Science/Technology 

> Mission Operations 


Program Management 


Risk 

Evaluation 

Remarks 

Technical 

Low 


Schedule 

Low 


Cost 

Low 


Resources 

Excellent 


Management 

Processes 

Excellent 



> Concerns: 


- None 


20 
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Aeronautical & Spacecraft Systems 


Risk 

Evaluation 

Remarks 

Technical 

Moderate 


Schedule 

Moderate 


Cost 

Moderate 


Resources 

Excellent 


Management 

Processes 

High 



> Concerns: 

- Software growth due to unexpected software problems 

- Baseline planning and earned value method used are not consistent with 
expected workload/ complexity. 

- Cost/ Schedule growth due to increase in software effort 

> Action: 

- Establish more rigorous management processes for baseline maintenance 

- Establish EV methods more appropriate to work accomplishment 

- Update EAC to reflect SW growth. 


Safety & Mission Assurance 


Risk 

Evaluation 

Remarks 

Technical 

Low 


Schedule 

Low 


Cost 

Low 


Resources 

Moderate 


Management 

Processes 

Excellent 



> Concerns: 

- Current resources are not accomplishing tasks in a 
timely manner. 


> Recommendation: 

- Assign proper staffing mix to S&MA tasks 
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Payloads 


Risk 

Evaluation 

Remarks 

Technical 

Moderate 


Schedule 

Low 


Cost 

Low 


Resources 

Excellent 


Management 

Processes 

Excellent 



> Concerns: 

- Current weight estimates above allowable weight. 

- Payload may require special support equipment for placement on 
the vehicle. 

> Recommendations: 

- Research methods for weight reduction 

° 23 

- Utilize COTS for support equipment. 


Systems Integration & Test 


Risk 

Evaluation 

Remarks 

Technical 

Low 


Schedule 

Low 


Cost 

Moderate 


Resources 

Moderate 


Management 

Processes 

Excellent 



> Concerns: 

• Budget for system level testing not identified. 

• Potential loss of staff and staff conflicts with testing 

> Action: 

• Identify budget for system Level testing. 

• Increase resources to meet test plan. 

r 24 
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Launch Vehicle/Services 


Risk 

Evaluation 

Remarks 

Technical 

Low 


Schedule 

Low 


Cost 

Low 


Resources 

Excellent 


Management 

Processes 

Moderate 



> Concerns: 

- No objective or documented quantifiable backup data 
for EVM methodology. None 


> Actions: 

- Identify and assign objective EV methods. 


Science / T echnology 


Risk 

Evaluation 

Remarks 

Technical 

Low 


Schedule 

Low 


Cost 

Low 


Resources 

Excellent 


Management 

Processes 

Excellent 



> Concerns: 


- None 


26 
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Mission Operations 


Risk 

Evaluation 

Remarks 

Technical 

Low 


Schedule 

Low 


Cost 

Moderate 


Resources 

Moderate 


Management 

Processes 

Excellent 



> Concerns: 

- Additional Ground Support Equipment may be required. 

- Unanticipated modifications to existing facilities to support processing. 

> Recommendations: 

- Ensure good configuration management planning for tracking design 
changes that may impact operations. 

- Identify additional costs in the EAC. 


Program Level Risks 


> Risks 

- Shared Resources with multiple programs 

- Software Slip 

- Software Maturity 

- Software Maintenance 

- Compressed Schedule 
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Actions / Recommendation 

>Govt EVM analyst to monitor MR usage for 
remainder of contract 

>Govt to send Contractor a request for an 
impact proposal 

- Impact of program schedule slips to software 
baseline 

- Schedule Review after contract modification to 
address the change to the Baseline 

29 


IBR Assessment 

> Strengths 

• CAM's knowledge 

• Cost, Schedule and Performance 

• EVMS 

• IBR preparation/ baseline documentation 

> Weaknesses 

• No significant weaknesses identified 



OVERALL ASSESSMENT 

GREEN 

30 
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APPENDIX L 


IBR LETTER OF FINDINGS TEMPLATE 



<Project Name> 

Integrated Baseline Review - Letter of Findings 

Supplier: < Supplier Name> 

Contract Number: <Contract #XXXXXXX> 


Date: <Month Day, Year> 
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To: <Supplier Contract Officer Name, Organization> 

SUBJECT: Notification of Integrated Baseline Review (IBR) Findings for <Subproject Name or Contract #>. 

The <Project Name> IBR Team conducted an Integrated Baseline Review (IBR) of contract <Contract #> 
at your facility in <Supplier Address> during <Date of IBR>. The team identified areas of concern, 
requests for documents and provided risk assessment as contained in attachment (1), Concern Area 
Reports. 

Status on action Items and documentation requested should be directed to <IBR Coordinator Name> at 
<Phone Number & Email>. Action Items and Documentation requests should be completed by <Date>, 
unless other arrangements are made with <Review Team Lead Name> at <Phone Number & Email>. 

Any contractual questions should be directed to <Project COTR Name> at <Phone Number & Email> and 
earned value management question should be directed to <Review Team Lead Name> at <Phone 
Number & Email>. 


<PM NAME> 

Project Manager 
<Project Name> Project 

Attachment 1: Concern Area Reports 
Cc: 

Government COTR 
Review Team Lead 
Review Team Members 
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APPENDIX M IBR REPORT 


Sample Review Report 


1. Introduction. Identify the contract purpose, type, duration, amounts (total, ceiling price, 
target costs, etc.), the project being supported, and the cognizant government component. 
Also, identify the specific contract requirement for an earned value management system. 

2. Purpose. Identify the purpose of the review. 

3. Scope. Identify the specific contractual entity that is the subject of this review; for 
example, division, company, plant, and the functional organizations, such as engineering, 
manufacturing, quality assurance, or individual process teams. Discuss whether the review is 
related to development, production, or construction contract. Identify CWBS areas covered, 
the methodology used in conducting the review, indicating such items as range of CAM 
discussions, depth of review, documents examined, and traces conducted. Team members 
and their associated responsibilities should be identified in this section. 

4. Findings. Identify areas of concern including a complete discussion of condition, cause 
and effect. If the concerns are not resolved by the time the report is written, a schedule for 
their resolution should be attached. 

During the course of the review, if concerns surfaced relative to the EVM System and its 
processes, these should be communicated to the appropriate personnel for proper resolution. 

5. Conclusions and Recommendations. This portion of the report contains any conclusions 
and recommendations based on review findings. This should include any action items and, if 
applicable, specific areas needing further review. 


135 


APPENDIX N 


IBR CLOSURE NOTIFICATION LETTER TEMPLATE 



< Project Name> 

Integrated Baseline Review -Closure Notification 

Supplier: < Supplier Name> 

Contract Number: <Contract #XXXXXXX> 

Date: <Month Day, Year> 
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To: <Supplier Contract Officer Name, Organization> 


SUBJECT: Notification of Integrated Baseline Review (IBR) closure. 

The <Project Name> IBR Team conducted an Integrated Baseline Review (IBR) of contract <Subproject 
Name or Contract #> at your facility in <Address> during <IBR Dates>. 

All actions resulting from Concern Area Reports from this review are closed. 

Any contractual questions should be directed to <COTR Name> at <Phone Number & Email> and earned 
value management question should be directed to <1 BR Team Lead Name> at <IBR Team Lead Name, 
Phone, Email>. 


<PM Name> 

Project Manager 
<Project Name> Project 
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APPENDIX O GLOSSARY 


GLOSSARY OF TERMS 

Actual Cost of Work Performed (ACWP) (or Actual Cost) . The costs actually incurred and 
recorded in accomplishing the work performed within a given time period. Actual costs include 
the direct cost plus the related indirect cost such as overhead, G&A, etc. allocated to the 
activity. 

Administrative Contracting Officer (ACO) . The individual within the Defense Contract 
Management Agency (DCMA) Contract Management Office (CMO) responsible for ensuring that 
the functions described in DFARS 242.302 are completed by the contractor in accordance with 
the terms and conditions of the contract. 

Applied Direct Costs . The actual direct costs recognized in the time period associated with 
the consumption of labor, material, and other direct resources, without regard to the date of 
commitment or the date of payment. These amounts are to be charged to work in-process 
when any of the following takes place: 

• When labor, material and other direct resources are actually consumed. 

• When material resources are withdrawn from inventory for use. 

• When material resources are received that are uniquely identified to the 
contract and scheduled for use within sixty days. 

• When major components or assemblies that are specifically and uniquely identified 
to a single, serially numbered end-item are received on a line-flow basis. 

Apportioned Effort . Effort which by itself is not readily divisible into short-span work packages 
but which is related in direct proportion to some other measured effort. 

Authorized Unpriced Work . Any effort for which contractually definitized costs have not 
agreed upon, but for which written authorization has been received. 

Authorized Work . Effort which has been definitized and is on contract, plus that effort for which 
definitized contract costs have not been agreed to but for which written authorization has been 
received. 

Baseline . See “Performance Measurement Baseline.” 

Bill Of Material (BOM) . A listing of material items required to complete the production of a 
single unit. When actual or expected prices are applied, it becomes the Priced Bill of Material 
(PBOM). 

Bottoms-Up Cost Estimate . An estimate derived by summing detailed cost estimates of the 
individual work packages and adding estimated level of effort plus appropriate indirect cost 
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estimates. Frequently accomplished as an independent analysis by Industrial Engineering, 
Price Analysis and Cost Accounting. 

Budget . A plan of operations for a fiscal period in terms of estimated costs or hours. 

Budget At Completion (BAC) . The sum of all budgets (BCWS) allocated to the project or a 
given Control Account. It is synonymous with the term Performance Measurement Baseline 
(PMB). 

Budgeted Cost For Work Performed (BCWP) (or Earned Value). The sum of the budgets for 
completed work packages and partially completed work packages, plus the appropriate portion 
of the budgets for level of effort and apportioned effort work packages. 

Budgeted Cost For Work Scheduled (BCWS) (or Planned Value) The sum of the budgets for 
all work packages, planning packages, etc., scheduled to be accomplished (including in-process 
work packages), plus the amount of level of effort and apportioned effort scheduled to be 
accomplished within a given time period. This is the value of planned work. 

Budgeting . The process of translating approved resource requirements into a time-phased 
plan for accomplishing work. 

Burden . See “Indirect Expense” and “Overhead.” 

Burdened Cost . The sum of direct cost plus all applicable indirect cost. 

Contract Budget Base (CBB) . The sum of the negotiated contract cost plus the estimated cost 
of authorized but unpriced work. IT includes the PMB and MR. Customer approval is generally 
required to change it. 

Contract Change Order . A formal revision or change to a contract that may impact the scope 
of work, schedule, cost, price, and/or other contractual requirements. 

Contract Data Reguirements List (CDRL) . A compilation of all data requirements (DD Form 
1423) that the contractor is obligated to submit to the government. 

Contract Line Item Number (CLIN) . The number used within a contract to identify a specific 
deliverable item. 

Contract Master Schedule (CMS) . The highest summary level schedule for a contract, 
depicting overall contract phasing and all major interfaces, contractual milestones, and contract 
elements (sometimes called Integrated Master Schedule). 

Contract Performance Report (CPR) . A contractually required recurring report designed to 
provide information on contract cost performance status, problems, and corrective actions (Dl- 
MGMT-81466). 

Contract Schedules . Schedules that are initiated at the direction of the customer or 
management to reflect program plans for development or production of deliverable items. The 
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highest level schedule is the Contract Master Schedule supported by Intermediate Level 
Schedules and by lowest level detail schedules. 

Contract Work Breakdown Structure (CWBS) . A work breakdown structure (WBS) of the 
products or services to be furnished under contract. It is comprised of selected Project WBS 
elements specified in the contractual document and the contractor’s lower level extensions of 
those elements. 

Control Account (CA) . A management control point at which budgets (resource plans) and 
actual costs are accumulated and compared to the earned value for management control 
purposes. A control account is a natural measurement point for planning and control since it 
represents the work assigned to one responsible organizational element (or integrated product 
team) for a single WBS element. 

Control Account Manager (CAM) . A manager responsible for task performance of a Control 
Account and for planning and managing the resources authorized to accomplish such task. 
Control Account Plan (CAP) . A format upon which a control account plan is displayed. A CAP 
typically displays the control account scope and budget in time-phased work packages and 
planning packages, cost element visibility, earned value techniques for each work package, 
responsible performing organizations and at least one charge number. 

Cost-At-Completion (CAC) . Actual direct costs, plus indirect costs allocable to the contract, 
plus the estimate of costs (direct and indirect) for authorized work remaining. The CAC is 
sometimes referred to as Estimate-at-Completion (EAC) or Latest Revised Estimate (LRE). 

Cost Element . Typical elements of cost are: direct labor, direct material, other direct costs, and 
indirect cost (overhead). 

Cost Incurred . A cost identified through the use of the accrual method of accounting and 
reporting or otherwise actually paid. Cost of direct labor, direct materials, and direct services 
identified to and necessary for the performance of a contract, and all properly allocated and 
allowable indirect costs as shown by the contractor’s books of record. (See Actual Direct Cost 
and Applied Direct Cost.) 

Cost Performance Index (CPI) . A measure of cost efficiency. It compares BCWP to the actual 
cost to perform that work (CPI=BCWP/ACWP). An index of 1.0 means that we are spending 
exactly what we planned to spend to accomplish the work performed. CPI > 1 .0 means we are 
under running costs. CPI < 1 .0 means that we are over running costs. 

Cost Plus Award Fee/Fixed Fee/Incentive Fee . Cost reimbursement contract types where the 
fee is based upon: (a) the accomplishment of pre-negotiated goals (Award Fee); (b) a 

negotiated fixed fee amount (Fixed Fee); or (c) a risk sharing ratio within a pre-determined 
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range based upon the contractor’s ability to meet technical, cost, and/or schedule targets 
(Incentive Fee). 

Cost Reimbursement Contracts . A category of contracts whose use is based upon payment 
by the government to a contractor of allowable cost as prescribed by the contract. Normally 
only the “best efforts” of the contractor are required. The basis for payment negotiated may be: 
(a) cost (no fee); (b) cost sharing; (c) cost-plus-fixed fee; (d) cost plus award fee, and/or (e) 
cost-plus incentive fee. 

Cost-To-Complete (CTC) Forecast . A contractor’s estimate of the cost to complete the 
remaining tasks on a contract. Synonymous with Estimate to Complete. 

Cost Variance (CV ). A metric for the cost performance derived from earned value data. It is 
the algebraic difference between the earned value (BCWP) and the actual cost incurred 
(ACWP). A positive value indicates a favorable condition and a negative value indicates an 
unfavorable condition. 

Critical Path . A sequential path of activities that are tied together network logic that have the 
longest overall duration from time now until project completion. 

Critical Subcontractor . A contractor performing a large or complex portion of a contract that 
requires a flow-down of earned value management and reporting (e.g., CPR) requirements, and 
the integration, reviews, acceptance and control of subcontractor system and reporting by the 
prime contractor. Critical subcontractors are designated as a result of customer negotiation or 
by management or by government direction. 

Defense Contract Management Agency (DCMA). The Department of Defense (DoD) 
component that works directly with Defense suppliers to help ensure that DoD, Federal, 
and allied government supplies and services are delivered on time, at projected cost, 
and meet all performance requirements. As the DoD Executive Agent for EVMS, DCMA 
is responsible for ensuring the integrity and application effectiveness of contractor 
EVMS. The NASA Program/Project contracting officer will normally delegate the 
responsibility for verifying a supplier’s initial and continuing compliance with ANSI/EIA - 
748 guidelines to the designated DCMA Administrating Contracting Officer (ACO) 
assigned to a DCMA Contract Management Office (CMO). 

Defense Contract Audit Agency (DCAA) . The DoD organization tasked with monitoring a 
contractor’s design and implementation of an acceptable accounting system. 
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Definitized . A contract, contract amendment, or contract supplemental agreement is 
considered definitized when the final contractual documents are unconditionally executed by 
both parties to the agreement. 

Direct Cost . Any costs that may be identified specifically with a particular cost objective. That 
is, the portion of labor, material or other cost incurred or expended to meet contractual 
specifications for an end product, tool or other related service specifically identifiable to the 
contractually authorized task. It consists of those costs that can be reasonably and consistently 
related directly and finally to the contract, without distribution through an overhead unit or 
account. 

Direct Labor . That portion of labor expended in the actual design, tooling, testing and the 
physical application of labor (including proofing) to material altering its shape, form, nature, or 
fulfilling a contractual requirement for service. 

Discrete Effort . Tasks which have a specific end product or end result, and which through 
planning: (1) can be specifically defined and assigned a budget for accomplishment; (2) can be 
scheduled with clearly definable start and completion dates; and (3) contain criteria against 
which performance can be measured. 

Discrete Milestone . A milestone which has a definite, scheduled occurrence in time signaling 
the finish of an activity, such as “release drawing,” “pipe inspection complete,” and/or signaling 
the start of a new activity. A type of “objective indicator.” 

Earned Hours . The time in standard or budgeted hours credited to a worker or group of 
workers as a result of their completion of a given task or group of tasks. 

Earned Value (EV) . The budgeted value for work accomplished. The value of completed work 
expressed in terms of the budget assigned to that work. It is the sum of budgets for completed 
work packages and completed portions of open work packages, plus the appropriate portion of 
the budgets for level of effort and apportioned effort. Also known as Budgeted Cost for Work 
Performed (BCWP). 

Earned Value Management (EVM). A tool for measuring and assessing project performance 
through the integration of technical scope with schedule and cost objectives during the 
execution of the project. EVM provides quantification of technical progress, enabling 
management to gain insight into project status and project completion costs and schedules. Two 
essential characteristics of successful EVM are EVM system data integrity and carefully 
targeted monthly EVM data analyses (i.e., risky WBS elements). 
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Earned Value Management System (EVMS) . The integrated set of policies, processes, 
systems and practices that meet an organization’s implementation of ANSI/EIA-748. An 
integrated management system and its related subsystems that allow for planning all work 
scope to completion; assignment of authority and responsibility at the work performance level; 
integration of the cost, schedule, and technical aspects of the work into a detailed baseline plan; 
objective measurement of progress (earned value) at the work performance level; accumulation 
and assignment of actual costs; analysis of variances from plans; summarization and reporting 
of performance data to higher levels of management for action; forecast of achievement of 
milestones and completion of events; forecast of final costs; and disciplined baseline 
maintenance and incorporation of baseline revisions in a timely manner. 

Engineering Change Proposal (ECP) . A proposed change, addition, or deletion to the basic 
contract, initiated by the contractor or customer. 

Engineering Release . A procedure or method for the formal issuance of initial or updated 
engineering data, instructions and drawings of detail parts, assemblies and components of a 
new or modified product. 

Estimate At Completion (EAC) . A value (expressed in dollars and/or hours) developed to 
represent a realistic projection of the final cost of a task (or group of tasks) when completed. 
EAC is the sum of direct and indirect costs to date, plus the estimate of all authorized remaining 
work. EAC=lnception to date ACWP+ETC 

Estimate To Complete (ETC) . A value (expressed in dollars and /or hours) developed to 
represent a realistic projection of the “to go” cost of the unaccomplished work to complete a 
task. 

Expenditure . A charge against available funds. It is evidenced by a voucher, claim, or other 
document approved by competent authority. Expenditure represents the actual payment of 
funds. 

Fiscal Year . For the United States government, it is the 12 month period: 1 October through 
30 September. For industry, it may be any formally selected annual accounting period including 
a calendar year. 

Fixed Price Contracts . A category of contracts based on the establishment of a price to 
accomplish the required work. Types are: (a) firm fixed price; (b) fixed price with escalation; (c) 
fixed price redeterminable, and (d) fixed price with incentive provisions. 

Front Loading . An action by a contractor to provide adequate or generous budget in the near- 
term budget baseline at the expense of the far-term effort. This practice delays 
acknowledgment of potential overrun conditions, frequently in the expectation that the contractor 
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can recover through subsequent changes in the contract statement of work. Front loading often 
results from an inadequate or unrealistic negotiated contract target cost and/or unrealistically 
optimistic planning of far-term effort. 

Functional Organization . An organization or group of organizations with a common 
operational orientation such as Engineering, Manufacturing, (Fabrication, Assembly), Tooling, 
Quality Control, Material, Finance, Contracts, etc. See Organization Breakdown Structure 
(OBS). 

Functional Manager . A line manager or supervisor of a functional organization. 

Funding Profile . A display of program funding requirements normally presented on a 
cumulative basis by months or quarters. Plotting of the requirements is displayed as a “stair 
step” chart. 

Gantt Chart . A horizontal bar chart. A graphic representation used as an aid to effective 
scheduling and control, showing graphically on a time scale when certain events are to take 
place or where deadlines occur. Status is displayed by either filling the bar or adding a filled-in 
status bar below it. 

General & Administrative Expense (G&A) . Expense incurred in the overall Corporate and 
Division offices, i.e., their cost of staff services such as, finance, legal, contract administration, 
sales, marketing and independent research and development (IR&D) effort. These expenses 
are allocated to organizational elements on the basis of total direct and indirect cost. 

Indirect Budget . The target value established for costs to be incurred by persons and/or 
organizational elements for tasks or expenses that do not have a direct relationship to the 
design, testing and/or production of the end product or contractually specified task. 

Indirect Cost . Costs that, because of their incurrence for common or joint objectives, are not 
readily subject to treatment as direct costs. This term is further defined in FAR. 

Indirect Cost Pools . A grouping of indirect costs identified with two or more cost objectives but 
not separately identified with any final cost objective. Such separate pools are normally 
established for indirect costs associated with Engineering, Manufacturing, Procurement, and/or 
Material, etc. 

Integrated Baseline Review (IBR) . A risk-based review conducted by the Program/Project 
Management to ensure a mutual understanding between the customer and supplier of the risks 
inherent in the supplier’s PMB and to ensure the PMB is realistic for accomplishing all the 
authorized work within the authorized schedule and budget. 

Integrated Surveillance Team (1ST ). Collectively the representatives of those organizations 
working together on the acquisition and administration of a contract or contracts. This could 
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include the PMO, DCMC, DCAA, the procuring activity integrated support components and the 
contractor. 

Inter Divisional Work Authorization (IDWA) . The document or procedure for “subcontracting” 
work between divisions of a company. Cost for authorized work by the performing division is 
transferred to the prime contracting division without G&A overhead and fee to preclude double 
charging on the prime contract. It is sometimes referred to as an Inter Group Work 
Authorization (IGWA) or as Inter Organizational Transfer (IOT). 

Interim Budget . Furnished to departments on an interim basis for contractually authorized 
tasks for which a firm bid or estimate may not yet have been completed and negotiated. It is 
also used for task transfers between departments or elements of cost pending formal budget 
transfer. Interim budget is discontinued after budget values are formalized. 

Internal Replanning . Replanning actions performed by the contractor for remaining effort 
within the recognized total allocated budget. 

Joint Surveillance . Project surveillance conducted jointly by the supplier and customer (e.g., 
for contractor’s this is typically performed by the contractor’s EVMS organization, the cognizant 
DCMA and the customer). 

Level Of Effort (LOE) . Effort of a general or supportive nature that does not produce definite 
end products. Examples include supervision, program administration and contract 
administration. 

Management Reserve (MR) . An amount of the total allocated budget withheld for management 
control purposes rather than designated for the accomplishment of a specific task or set of 
tasks. It is not a part of the Performance Measurement Baseline. 

Master Planning Schedule (MPS) . The highest summary level schedule for a contract 
depicting overall contract phasing and all major interfaces, contractual milestones, and program 
elements. (See Contract Master Schedule.) 

Material . Property which may be incorporated into or attached to an end item to be delivered 
under a contract, or which may be consumed or expended in the performance of a contract. It 
includes, but is not limited to, raw and processed material, parts, components, assemblies, fuels 
and lubricants, and small tools and supplies. 

Milestones . Events of particular significance. Finitely defined events that constitute the start or 
completion of a task or occurrence of an objective criterion for accomplishment. Milestones 
should be discretely identifiable; the passage of time alone is not sufficient to constitute a 
milestone. However, milestones should be associated with schedule data to document when 
the milestone is to occur. (See Objective Indicator.) 
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Negotiated Contract Cost (NCC) . The estimated cost negotiated in a cost-plus-fixed-fee 
contract or the negotiated contract target cost in either a fixed-price incentive contract or a cost- 
plus-incentive-fee (or award fee) contract. 

Network Schedule . A schedule format in which the activities and milestones are represented 
along with the interdependencies between activities. It expresses the logic as to how the 
program will be accomplished. Network schedules are the basis for critical path analysis, a 
method for identification and assessment of schedule priorities and impacts. 

Non-Recurring Cost . Expenditures for specific tasks that are expected to occur only once, or 
very infrequently, on a given program. Examples are such costs as those for preliminary design 
effort, qualification testing, initial tooling, testing, planning, etc. 

Objective Indicator . A finite event or accomplishment that can be used to definitively establish 
the degree of completion of a specific task. 

Organizational Breakdown Structure (OBS) . The project hierarchy of line and functional 
organizations as applied to the specific project. 

Other Direct Cost (ODC) . A group of accounting elements, other than direct labor and 
material, which can be isolated to specific tasks. ODC includes such items as travel, computer 
time, aviation fuel, etc. 

Overhead . (See Indirect Costs.) 

Overrun . Costs incurred in excess of the allocated budget for a task, group of tasks, or a 
contract. For work in process, the overrun is the cost incurred in excess of earned value. 

Over Target Baseline (OTB) . Replanning actions involving establishment of cost and/or 
schedule objectives that exceed the desired or contractual objectives on the program. An OTB 
is a new baseline for management when the original objectives cannot be met and new goals 
are needed for management purposes. 

Performance Measurement Baseline (PMB) . The time-phased budget plan against which 
contract performance is measured. It is formed by the budgets assigned to scheduled control 
account and the applicable indirect budgets. For future effort, not planned to the control 
account level, the performance measurement baseline also includes budgets assigned to higher 
level WBS elements and undistributed budgets. It equals the total allocated budget less 
management reserve. 

Performing Organization . A defined unit within the contractor’s organization structure, which 
applies resources to perform the work. 

Planned Value . Same as Budgeted Cost for Work Scheduled. 
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Planning Package (PP) . A logical aggregate of far-term effort within a control account that can 
be identified and budgeted, but not yet defined into discrete Work Packages. 

Post Acceptance Review . A government review performed on a specific element or elements 
of a contractor’s EVMS system that display(s) a lack of discipline or no longer meet(s) the intent 
of the EVMS Criteria. 

Priced Bill-Of-Material System . An automated requirement generation system for material 
and parts, which lists all items, quantities, sources, descriptions and prices in an indentured 
listing, which relates a part of component to a higher or using assembly. Such a system may be 
used for pricing material costs for proposals. 

Price Variance (PV) . The portion of a material cost variance due to price change. The 
difference between the planned unit cost of materials and the actual unit cost of material. PV is 
derived by subtracting the planned unit price times the quantity used from the actual unit price 
times the quantity used. 

Problem Analysis Report (PAR) . A report made by the responsible manager to explain a 
significant cost or schedule variance, its probable impact on the program, and the corrective 
action(s) taken or required to resolve the problem (Same as Variance Analysis Report). 
Procuring Activity . The subordinate command in which the Procuring Contracting Office 
(PCO) is located. It may include the Program Office and other related functional support 
activities. 

Program (or Project or Product) Manager (PM) . The person assigned the prime 
responsibility for overall management of a program (or project, or product). 

Program Master Schedule . The highest summary level schedule for a major program 
depicting overall program phasing and interfaces, contractual milestones, and major events that 
support specific program objectives. 

Program Work Breakdown Structure (PWBS) . See part (a) of Work Breakdown Structure 
definition. 

Progress Payments . Payments made to a contractor or subcontractor during the life of a fixed 
price type (firm fixed price and fixed price incentive) contract on the basis of a percentage of 
total incurred cost. 

Purchase Order . An order issued by a functional organization (usually material or purchasing) 
to purchase parts, supplies or services from an outside source. 

Purchase Reguest (PR) . A contractor’s authorizing procurement document that results in the 
issuance of a Purchase Order. Also referred to as a Purchase Order Request (POR). 
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Purchased Labor . A type of labor used on a contract basis to relieve an engineering or shop 
overload and/or to take advantage of special processing or technical skills or of special facilities 
possessed by a supplier. 

Purchased Parts . Detail parts or small subassemblies that are purchased from or 
subcontracted to an outside source. Such parts or subassemblies are purchased to relieve a 
shop overload or because they are not within the prime contractor’s normal capability to make. 
Quantity Variance (QV) . (See Usage Variance). 

Risk Analysis . The system that provides a continuous analysis of identified risks with respect 
to their impact on program cost, schedule, and technical performance. 

Realization Factor . The ratio of actual performance time to standard performance time, usually 
expressed as a decimal number. 

Recurring Costs . Expenditures against specific tasks that occur on a repetitive basis. 
Examples are costs of sustaining engineering support, repeated fabrication or assembly of parts 
or products, tool maintenance, etc. 

Replanning . The process by which a program or project updates or modifies its plans. This 
applies to a change in the original authorized PBB or CBB planning for accomplishing formally 
authorized requirements, typically involving the redistribution of budget for remaining work. In 
accordance with the ANSI/EIA-748, traceability is required to previous baselines, and funding 
requirements need to be considered in any replanning effort. There are two types of replanning: 

• Internal Replanning. Replanning actions performed by the supplier for remaining effort within 
the recognized Project Budget Base or CBB. It is caused by a supplier’s need to 
accommodate cost, schedule, or technical problems that may have made the original plan 
unrealistic. Internal replanning is restricted to remaining effort and if significant, the customer 
must be advised of the action. 

• Authorized Change (or External) Replanning. A change necessitated by government/ 
customer direction which may be in the form of either a definitized or a no cost contract 
change order for contracts or formal change to the Project Plan for in-house Projects that 
calls for a change in the original plan. It most often results from a change in the authorized 
requirement affecting cost, schedule, technical parameter or a combination thereof. 

Reprogramming (or Formal Reprogramming) . A comprehensive replanning of the remaining 
PMB that result in an Over-Target Baseline (OTB), an Over-Target Schedule (OTS) or both. 
This type of replan is for performance measurement purposes only and requires prior 
coordination and approval of the Customer. 
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Responsible Organization . A defined unit within the contractor’s organization structure that is 
assigned responsibility for accomplishing specific tasks. (See Performing Organization for 
comparison.) 

Responsibility Assignment Matrix (RAM) . A chart showing the relationship between the 
CWBS elements and the organizations assigned responsibility for ensuring their 
accomplishment. The RAM normally depicts the assignment of each control account to a single 
manager, along with the assigned budget. 

Rolling Wave Planning . The progressive refinement of work definition as time goes on by 
continuous subdivision of downstream activities into detailed tasks. 

Rubber Baselining . Actions by a contractor to advance far-term budgets into the current or 
early periods to mask current cost problems. The action involves moving budget without a 
corresponding amount of task, to cover current cost difficulties. It is an indication of likely 
overrun condition. 

Schedule . A plan that defines when specified work must be started, worked on, and finished, to 
accomplish program objectives on time. 

Schedule Performance Index (SPI) . A measure of schedule efficiency. It compares the 
BCWP to the work scheduled (SPI = BCWP / BCWS). An index of 1 .0 means the work is being 
performed right to the schedule. SPI > 1 .0 means that the work is ahead of schedule. SPI < 1 .0 
means that the work is behind schedule. 

Schedule Variance (SV) . A metric for the schedule performance on a program. The algebraic 
difference between the earned value (BCWP) and the budget plan (BCWS). (SV=BCWP- 
BCWS) A positive value is a favorable condition while a negative value is unfavorable. It may 
be expressed for a specific period or cumulative to date. 

Significant Variance . Those differences between planned and actual performance that require 
further review, analysis, and/or action. 

Statement Of Work (SOW) . The document that defines the work scope requirements on a 
program or contract. 

Subcontract . A contract for services, data, parts, components, assemblieSi other hardware* or 
software that a company commits to perform for or provide to the prime contractor. A 
subcontract normally involves the design or production of a component by the supplier to the 
prime contractor’s specifications. “See Purchased Parts.” 


149 



Summary Level Planning Package (SLPP) . An aggregation of work for far-term efforts, not 
able to be identified at the control account level, but which can be assigned to higher level WBS 
elements (and is therefore not “undistributed budget”). 

Surveillance . The continuous process of reviewing the health of the earned value 
management system (EVMS) as applied to one or more projects. 

Surveillance Plan . An annual plan that identifies the projects to be included in surveillance 
reviews, as well as the frequency and scope of the individual surveillance visits planned for 
each project included in the annual plan. 

Task . A piece or portion of discrete, apportioned, or level-of-effort work. Also called an activity, 
something that takes place over a period of time and generally consumes resources. 

Task Authorization . A document used in some contractor systems in conjunction with a 
Resource Authorization Document (RAD) to provide detailed work instructions and other 
technical information. 

Thresholds . Boundaries or limits (monetary, time, or other values), which, if breached, result in 
some type of management review and action. 

To-Complete Performance Index (TCPI) . The future cost efficiency needed to accomplish the 
remaining work within a financial goal such as the Budget at Completion (BAC) or the Estimate 
at Completion (EAC). It compares the budget for remaining work with the remaining cost or the 
estimated remaining cost to complete the work. TCPIBAC = (BAC - BCWP cum) / (BAC - 
ACWP cum). Or TCPIEAC = (BAC - BCWP cum) / (EAC - ACWP cum). Compare the CPI to 
determine if the BAC or the EAC is realistic or not. 

Total Allocated Budget (TAB) . The sum of all budgets allocated to a project/contract. Total 
allocated budget consists of the PMB and all MR. The TAB should reconcile directly to the 
project/contract budget base. If the TAB is greater than the project/contract budget base, the 
difference is attributable to an over target baseline and must be documented. 

Touch Labor . Production labor that can be reasonably and consistently related to a unit of 
work being manufactured, processed or tested. Also referred to as “Hands On” labor. 

Undistributed Budget (UB) . Budget associated with specific work scope or authorized 
changes that have not been assigned to a control account or lower level WBS element. 

Unit Cost . Total labor, material, and overhead cost for one unit of product, i.e., one part, one 
end item etc. 

Unpriced Changes . Authorized contract changes that are not yet priced. 

Usage . The number of units or dollar value of material or items used over a period of time. 
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Usage Rate . A percentage, based on historical records, applied to the Priced Bill of Material to 
accurately budget for scrap, breakage, lost in shop, rework design, error and surplus material 
consumed on a project or contract. 

Usage Variance (UV) . The portion of material cost variance due to a change in quantity of 
material used. The difference between planned quantity of materials and the actual quantity 
used, expressed in dollars. UV is derived by subtracting from planned quantity times planned 
unit cost, the actual quantity times planned unit cost. (Same as Quantity Variance.) 

Variance Analysis Report (VAR) . An internal document, within an earned value management 
system, for the analysis and reporting of variances which breech the established thresholds. It 
requires the reason or cause of the variance; the impact on cost or schedule; and the corrective 
action required, or accomplished, on significant variances. 

Variance At Completion (VAC) . The difference between the total budget assigned to a 
contract, WBS element, organizational entity or control account and the estimate at completion. 
It represents the amount of expected overrun or underrun. (VAC equals BAC minus EAC, i.e., 
total allocated budget minus the related estimated final cost). 

Variance Threshold . Internal and external tolerances (or thresholds), which are established by 
management direction, or negotiation with the customer and which, when exceeded, require 
investigation, analysis, reporting and corrective action. 

Work Breakdown Structure (WBS) . A product-oriented hierarchical breakdown or division of 
hardware, software, services, and other work tasks that organizes, displays, and defines the 
products to be developed and/or produced and relates the elements of the work to be 
accomplished to each other and the end product(s). 

Work Breakdown Structure Dictionary . A document that describes the tasks-associated with 
each WBS element, in product-oriented terms, and relates each element to the respective, 
progressively higher levels of the structure as well as to the contract Statement of Work. May or 
may not be a contractual requirement. 

Work Breakdown Structure Element . A single discrete portion of a WBS. May be an 
identifiable product, a set of data, or a service. 

Work Package (WP) . A detail, short duration task or material item identified by the Project 
Control Account manager for accomplishing a Control Account task. A work package has the 
following characteristics: 

• Represents unit of work at the level where work is performed. 

• Clearly separate from other Work Packages. 

• Assignable to a single organizational element. 
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• Has scheduled start and completion dates, and interim milestones, if required, all of which 
represent physical accomplishment. 

• Has budget expressed in terms of dollars or hours/FTEs. 

• Its duration is limited to a relatively short span. 

• Is integrated with detailed engineering, shop, or other schedules. 

• Has a correct Earned Value Technique assigned to it. 

Work Package Budgets . Resources that are formally assigned by the contractor to 
accomplish a work package, expressed in dollars, hours, standards or other definitive units 
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APPENDIX P 


CxP CANDIDATE RISK ASSESSMENT FORM 



CxP Candidate Risk Assessment Form 



Candidate Risk: The IBR Team member will identify a candidate risk 
based on a combination of the likelihood of an undesired event. . .and the 

Date: 

consequences if the event were to occur to the Constellation Program’s goals 
and objectives as related to Technical, Cost, and Schedule. 

Classification: 

Technical (Yes or No): 


Cost (Yes or No): 


Schedule (Yes or No): 


Owning Organization: 


Condition Statement (a phrase briefly describing current key circumstances, situations, etc. that are causing concern, 
doubt, or anxiety): 


Consequence Statement (a phrase that describes the key, negative outcome of the current condition): 


Context (any additional information to support your identification of this risk): 


IBR Team Member Signature: 
Print Name: 


IBR Team/Sub-team Lead Signature: 
Print Name: 


IBR Risk Management Officer Signature: 
Print Name: 
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